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Foreword 


This handbook aims to be accessible to the major 
public. This work is meant to be used as a pedagogical tool 
and as a reference book. The following book represents 
personal reflections on the probabilistic nature of measures 
in science. In a classical curriculum these aspects are 
seldom, if not at all, dealt with. It is important that the 
experimental and practical foundations of science are 
complementary to the theoretical lectures. There is a 
scientific beauty that arises from the interaction between 
theory and experience. 

While introducing the fundamental principles of 
statistics, this book explains how to determine uncertainties 
in different experimental situations. Many examples come 
from courses and practical work done in preparatory 
classes for the engineering schools 


T hope you enjoy reading! 


Thanks to the readers who by their questions, comments 
and constructive criticisms make it possible to improve the 
book. 


Thanks to the life and to all those who have come before 
me. 
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I. RANDOM VARIABLE 


A. How to measure a quantity ? 


In what follows, X is a random variable and {x:} a 
sample of n outcomes. 


If we ask how many days are in a week, it is easy to 
answer. Now, consider different groups of students who 
have measured the thermal capacity of water’ and have 
obtained the following values : (5100; 4230; 3750; 4560; 
3980} J/K/kg. How can we estimate the water capacity in 
this situation? The answer will use a probabilistic approach. 


B. The center of a distribution 


How does one determine the most representative value 
of a sample? There are different ways to define the center 
of a distribution. For example, we have the mode, the 
median and the mean, The mode is the value which occurs 
with the greatest frequency in a data set. The median is the 
middle of a distribution that divides the sample into two 
parts whereby each half has an equal number of 
observations, The most commonly used measure of the 


Inaysique : The amount of energy needed to raise the temperature of 
one kilogram of mass by 1°C. That way, the water stores energy 
and can then return it by decreasing its temperature. Tables : Cys 
4180 Joules per Celsius degree and per kilogram. 
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center is the mean, The mean is the sum of the observed 
values divided by the number of observations : 


For the thermal capacity of water we have : 


= 510044230 + 3750+4560+3980 _ 494 5/K/kg 


We have considered the arithmetic mean. We could have 
used the geometric mean: 


For instance for two speeds of 20 m/s and 40 mis, the 
geometric mean is \20m/s-40m/s~28.3m/s whereas 
the arithmetic mean is of 30 m/s. The arithmetic mean is 
used more often globally due to its conveniently simpler 
calculation. 


2Quari : To simplify the writing of a sum, the Greek letter sigma is used 
as a shorthand and read as "the sum of all x i with i ranging from 1 


ton" 


C. The dispersion of a distribution 


In addition to locating the center of the observed values we 
want to evaluate the extent of variation around the center. 
‘Two data sets may have the same mean but may be differ- 
ent with respect to variability. There are several ways to 
measure the spread of data, Firstly the range is the differ- 
ence between the maximum and minimum values in the 
sample. The sample range of the variable is very easy to 
compute, however it is sensitive to extreme values that can 
be unrepresentative. 


‘The sample standard deviation is preferred: 


Itis the most frequently used measure of variability. 


For the thermal capacity of water we have : 


| (5100-4324 )?-+{ 4230-4324 )*+ 
S.Sy 


(3750—4324)-+{ 4560—4324)-+(3980—4324)° 
a 


301K kg 


‘The mean deviation may also be used 


Using the standard deviation formula, 


root of average square deviation). 


(see Exercise 1). 


. dividing by n rather 
than n-/, will obtain the root mean square deviation (square 
The choice of the 
standard deviation is justified on page 128. Besides, the 


values of n are often large and the difference is small. 


D. Examples of distributions 
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‘The mean is not always the most represented value (case I 
and 2) and in some cases does not appear at all. In case 3 
the histogram is symmetrical and illustrating that the 
median and mean are equal. 


In the event that some values are represented several times, 
we determine the frequency f, for each value x, 


We have n= >> f,, where c gives the number of different 
i 


values of xi. 


The mean and standard deviation then become: 


Sometimes the data can be grouped into class intervals. For 
example, if we measure the size of the inhabitants of a city, 
we can group all the inhabitants with a size between 160 
cm and 170 cm within the same class interval. The number 
of observations in this class is the frequency and the middle 
of this class interval is the value assigned, here 165 cm (see 
Exercise 5). 


The more the histogram is concentrated around the center, 
the more the standard deviation is small. 


E. Central limit theorem 


1) Population and samples 


Consider a city of one million inhabitants. To survey 
the population we can interview a sample of only one thou- 
sand people drawn at random. Thanks to the statistical 
tools, from this n=1000 individuals sample, we can have 
information on the whole population. The larger the sample 
size is, the more accurate the results will be. Let X be the 
sample mean and s be the sample standard deviation. For 
the population we denote jt (Greek letter mu) the mean 
and o (sigma) the standard deviation. The larger the sam- 
ple, the more likely X and s are close to ju and o respec- 
tively. 


In the case of opinion polls, samples are around one 
thousand people. If we measure the size of one thousand 
inhabitants selected randomly from the population of a city 
of one million people, the average size of this sample is 
likely to be close to the average size of the entire popula- 
tion but has no reason to be equal. 


Let us take the example of a coin toss. The coin is 
balanced and the outcomes are heads or tails. In this case 
the population is infinite and we can have an infinit number 
of measurements. Furthermore, the probabilities are known 
and we can determine the population features. 


When the sample size becomes very large, it tends towards 
the population: =lim x 


We introduce here the concept of probability : 


im where ps the probability of the outcome x. 


Esra 


With this formula (using the formula page 6) we find the 
population mean formula: K= >> PX, . 


Also if we consi 


Yp=i as 


ler all the events possible, we have 
00%). 


The outcome heads is associated with x:=0, and the out- 
come ‘ails with x=/. The coin is balanced as 
po=pi=1/2=0,5=50% and 1=po.ro +p. Xi. 


Furthermore: o=lim sand with the formula for s page 6 


we obtain o=)D) p(x —n) (for m large, n-T is close to 


n). 


Eventually: #=0,5 and o=0,5 . 


3. sari : Reads as «is equal to the limit of ¥ when m tends to 
infinity». 


Let us flip nine coins and collect a sample : 
{Os 1s 15.05 15 15 150; 0). 
Then we find ¥~0,56 and s=0,53 . 


If this procedure is performed many times, each time we 
would have a different result for x . 


For example, if two other samples are taken: 
{1: 150; 1; 1; 0; 1; 1,0} then X=0,67 and s=0,50 
~0,44 


(Called the sampling distribution) 


‘The values obtained for the samples are generally different 
from those of the population, but the larger the sample, the 
more likely it is that the values are closer to those of the 
population. 


Case of a sample with n=50, where X=0,520 and 
s=0,505 : 


(001000111111100101 11110001001 1011010001 10000011101 } 


2) The central limit theorem 


|cENTRAL LIMIT THEOREM : 


Within a population we collect random samples of size n. 
The mean of the sample X varies around the mean of the 
[population 4. with a standard deviation equal to on, 
where o is the standard deviation of the population. 

|4s n increases, the sampling distribution of x is} 
lincreasingly concentrated around 1 and becomes closer and| 
|closer to a Gaussian distribution. 


We will describe in due course what a Gaussian distri- 
bution, also called normal distribution, is. For the moment 
we will simply consider a bell curve. This is a very impor- 
tant theorem. Whatever the form of the population distri- 
bution, the sampling distribution tends to a Gaussian, and 
its dispersion is given by the Central Limit Theorem. 


This is illustrated through the following diagrams: 


i : 
sample 
Is ofsten 
/o, 5 
nae = x 
; es t+ 


On the left we have the probability p of an event x (popula- 
tion distribution). 
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Hypothetically, for a city with a population of one million 
inhabitants, p could represent the probability that one have 
a given height x. If we could measure the height of all the 
inhabitants, we could exactly determine their average 
height 1 and the standard deviation «. However, it is prag- 
matically difficult, if not impossible, to measure a popula- 
tion of that size. Therefore a sample size of only a thousand 
inhabitants is taken to reduce the burden of labour. For this, 
to be as representative of the whole population as possible, 
the thousand-person sample is picked randomly. 


We obtain a thousand measures of height from x7 to xis. 
From this sample of size n=/000 we calculate a mean X 
and a standard deviation s, We think that X is close to j1, 
but at the same time there is no reason for it to be equal to 
4. We put this value of X on the right side of the figure on 
page 10. 


We take a new random sample of a thousand people and a 
new value for X . 


We then repeat this operation a great number of times. We 
see on the right the distribution of the samples obtained: 


B pi Gaussian 
samples 
of size n 
- x 
aS Fa 
=z, 
+ rer: + 
populati ‘sampling distribution 
distribution of the mean 


i 


3) Student's t-value and uncertainty 


The central limit theorem applies to the limit of large 
numbers. In the particular case where the distribution of 
the population is normal we can apply it from n small 
thanks to the coefficients of Student 1. 


Prediction interval : 


If and o are known, the sampling distribution is also 
Gaussian and the expected statistical fluctuations are with a 
probability of p% between t—t,,o/\/n and w+t,o/Vn. 


‘The t-values are read on page 177. 


Confidence imerval : 


In the case of the calculation of uncertainties. and o are 
not known and we estimate them from the sample with % 
and s. Due to a small statistic, there is widening given by 
the Student's t-distribution: 


‘The Student's ‘-value depends on the sample size n and on 
the confidence. If the confidence is 95%, we have 95 in 100 
chanee that wis between X-t-s/Vn and X+t-s/Vn. 
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We recognize here the notion of measurement uncertainty 
Axt 


x=x+Ax with Ax= 


yn 


A. is also called absolute uncertainty and A x/|%| rela- 
tive uncertainty. 


Let us take again the calorimetry experiment described on 
page I. We want to know the thermal capacity of the water 
with a confidence of 95%. As is often the case in experi- 
mental sciences, we consider that the data follow a normal 
distribution, because by the influence of many independent 
factors on the value of the measured quantities, we still ex- 
pect, under the central limit theorem, to have Gaussian 
fluctuations. 


We find for four degrees of freedom (ddl=n-/) a Student's 
tof 2.78. 


From where : 
95% confidence. 
Here following the dispersion of the values measured by 
the students : Ac/?~15%. The calorimetry measure 
ments are usually imprecise. The expected value, known 
here, is well within the range : 


3660<4180<4980 


+t-s,/Vn=4320+660J/K/kg with 


In experimental sciences we endeavor to quantify all 
natural phenomena. Yet, due to the very nature of the ex- 


mari : Reads "delta x" 
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perimental approach, the various parameters which make it 
possible to describe a experimental situation are not per- 
fectly known. We do not have a simple numerical value as- 
sociated with each characteristic, but an interval for a given 
confidence. Strictly speaking, any experimental value must 
associate its uncertainty with its confidence. 


Exceptions: 


» Large number samples: the size of the sample 7 is large 
enough to be able to directly apply the central limit theo- 
rem. The sampling distribution is normal regardless of the 
distribution of the population. We do not have to worry 
about Student's distribution which anyway identifies with a 
Gaussian distribution in this case. 


» Small samples and normality of the data: we apply the 
Student law as before. 


» Small samples with non-normality of data: For example, 
we find by computing the skewness and kurtosis of the data 
that they do not match a normal distribution. To counter 
this, it is necessary to perform a case by case study. For in- 
stance, when we have a uniform distribution of the popula- 
tion, the prediction interval given on page 12 works from 
n=2 (it is shown by using the data of the article Measure 
with a ruler p151) . However for a binomial distribution 
with parameters n=2 and p=0.24, the 50% prediction inter- 
val contains 0% of the values... A more complex case on 
page 144 shows for n= 12, in comparison with a numerical 
simulation, that the central limit theorem underestimates 
the confidence interval. 
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4) Examples 


‘A large number of random factors will influence the 
measurement of a physical quantity; independent factors, 
which, whatever their natures will ultimately generate a 
Gaussian distribution of observations. Let us consider two 
examples, the toss of a coin and the roll of a six-sided die. 


Consider the sample space for tossing a fair coin n 
times. We count the number of sails. For one toss, we have 
two outcomes possible, one with zero tails and one with 
one ‘ails. 


For two tosses, we have four outcomes possible, one with 
zero tails (H H), two with one tails (H T or T H) and one 
with two tails (T T). The more tosses that are thrown, the 
closer we get to a Gaussian distribution. 


= toses TY 
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We can obtain the probability (the number of tails divided 
by the number of possibilities 2") as a function of the 
number of tails. For n = 1 we have the distribution of the 
population and following the sampling distributions for 
different values of n. 


‘ 7 

6 

: - 5 

THH TTH 4 

2 | 3 

HTH THT 2 

three * ieeed| four 1 

tosses. HHHHHTHTT TIT tosses 0 
See Fae aS 01234 

2 25 

10 e 

® 15 

6 

‘io 10 

five 2 six 5 

tosses 0 EE Fl tosses 0 
012345 0123456 


Similarly for the dice we enumerate the possibilities for 
their sum and the distribution also tend towards a Gaussian. 
For a single die, the sum simply corresponds to the value of 
the die. We have a possibility for each value: 
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one die : z 


For two dice, there is only one possibility for the sum to be 
two: | for the first die and 1 for the second die. For the sum 
to be three, there are two possibilities: 1 and 2, or, 2 and 1. 
‘The most likely with two dice is to obtain a sum of 7: (1.6) 
(6,1) (2,5) (5,2) (3.4) (4,3). 


two dice : 4 


three dice: * 
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For four dice, we already recognize the bell curve and the 
profile is clearly of the Gaussian type: 


dice : ——— 
45 6 7 8 9 10111213 14.15 16 17 18 19202122 28.28 


On this last example, we check the validity of the central 
limit theorem. 
‘The population mean is : 
ft, =(14243+445+6)/6=3.5 
We verify that the sampling distribution mean is the same: 
H=14/6=3.5 
‘The population standard deviation is : 
Spa then 


T/6|(1-3.5) 12-35) +3-3.5) 44—3.5) 415-35] +16-3.5) | 


and @,=171 


also for four dice: 0,=0,/Vi=0,/2~0.85 . Now, on the 
above curve, 40% from the maximum (explanation page 
21), we have a deviation close to 3.5 (between 3 and 4), 
and an average of 3,5/4~0.88 . It matches. 


F. Gaussian distribution 


1) Definition of continuous distribution 


Some quantities are fundamentally continuous : time, 
space, temperature, et cetera. Time is like fluid, it does not 
jump from one value to another. A continuous random 
variable can take any value in an interval of numbers and 
has a continuum of possible values. On the other hand 
when we throw a six-sided die, it is impossible to say "I got 
2.351". It is a forbidden value, only integer values from one 
to six are allowed. 


Thus, 
others continuous. For a die we have : p) 


some probability distributions are discrete and 
ps=1/6 and 


> p.=1. Now, if we are interested in the height of the 
a 

inhabitants of a city, there is a continuum of possibilities, 
the distribution is continuous. We define the probability 
density function p(x) with p(x)dx the probability to be be- 
tween x and x+dx. Where dx is a small variation, and x+dx 
is close to x. The continuous random variable probability 
model assigns probabilities to intervals of outcomes rather 
than to individual outcomes. 
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So, the probability that the event is realized on the set of 
possible values is 100%: 


f placer 


Mean and variance in the continuous case : 


=f x pla) V=0" Flu plaid 


2) Bell-shaped density curve 


‘A continuous random variable X following normal distribu- 
tion has two parameters: the mean y and the standard devi- 
ation o. Density function : 


V21-o 


In the mathematic tools section page 165 some demonstra- 
tions are performed. 


5 ari : The mean is also called E(X), expectation of X. o°=V(X) is 
called variance of the random variable X. Properties 
B(aX+b)=aE(X)+b, VaX)=a°V(X) and V(X)=EOC)-EQ0* 
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We have represented two cases on the following graph: 


Gauss 
curves 


‘The total area under the curve is always 1. The probability 
concentrated within interval [11 — 0, t + 0] is 0.68 : 


J p(x)dx=0.683...~68% 


We evaluate the standard deviation at 60%Ps: 


P(t)! Pyg=1/Ve~0.607 


The probability concentrated within interval [jt — 20, + 
2a] is 0.95 : 


ws20 


J plx)dx=0.954...95% 


The probability concentrated within interval [jt — 30, + 
3a] is more than 0.99 : 


eae 


J p(x)dx=0.997...>99% 


3) Standard normal distribution 


A. standard normal distribution Z is normally distributed 
with a mean 4 = 0 and a standard deviation 6 = /. 


The distribution X is transformed into a distribution Z 
using the following two transformations : x’=x—\ 


and 


G. Hypothesis test 


Similarly to how to estimate the mean of an unknown 
probability distribution, the central limit theorem is also 
used for a hypothesis test. With a collected sample we have 
used the properties of the sampling distribution to 
determine a value and it’s uncertainty. For the hypothesis 
tests, we proceed in the other direction: the law of 
probability is assumed known, so the sampling distribution 
is perfectly known and we take a sample to define a 
decision criterion allowing us to accept or reject the 
hypothesis. 

Using the mean we test a hypothesis Hy. ‘This is re- 
ferred to as the null hypothesis. It is an assumption made 
on the probability distribution X. Let jt and be the mean 
and standard deviation of X. We take from the population a 
sample of size n large enough. The sample mean is x. If 

X is between u—t,.o/Vn and «+t,.o/Vn then Hy is 
accepted. However if x is outside of those values, the null 
hypothesis is rejected (two-tailed test). 


Population distribution Sampling 
Fe X known : pa distribution : 
Hy 
Central 
Limite 
Theorem 
(oy — 
rm z is E ® 
Teston X = —— 7a 
Hy ae XaeeeKaeak Saget a tes te 
sample "rejected accepted rejected 


We consider the coefficient t. of a normal distribution for a 
p% confidence (or Student's t-value when n > 2). 


We can also use other characteristic intervals of the sam- 
pling distribution. In general, the hypothesis Ho implies a 
property A of the sampling distribution. Here, the involve- 
ment is not deterministic but statistical and decision-mak- 
ing proceeds differently. 


Deterministic test case: Hj A 
+ If we observe A then Hy cannot be rejected. 


* If we do not observe A then Hp is rejected’. 


Statistical test case : 


In p% of cases: Hy A 


In (1-p)% of cases : 


HyeA 
+ If we observe A then Hy cannot be rejected 
with p% confidence. 


+ If we do not observe A then Ho is rejected, with a 
risk to reject Ho when it is true of (1-p)%. 


This protocol makes it possible to make a decision, but at 
the same time it carries risks of making a mistake. We can 
reject the hypothesis when it is true or we can accept the 
hypothesis when it is false. 


6 The contrapositive of an implication : if PQ then Q=>P 
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Accept Ho Reject Hy 


Wrong dk 


Right decision | (error of the first 
kind a) 


jon 


Wrong decision 
Hy is false (error of the Right decision 
second kind p) 


The aim is to minimize the risks linked to a and B. If we 
accept Hs, then at (the probability of rejecting the null hy- 
pothesis when it is true) has to be important, and B (the 
probability of accepting the null hypothesis when it is false) 
has to be low. If, on the contrary, we reject Hy it is because 
cis low and f important. 


When H, is true we are able to compute the risk, and a is 
equal to I-p. However if Ho is false we cannot compute B, 
unless an alternative hypothesis H, is known. 


For a standard test we fix I-p in advance. For example we 
can consider the test statistically significant at the threshold 
of 5%, and, according to the result found reject or accept 
the hypothesis, Another method is to calculate the 
probabilities « and which correspond to the value X 
found with our sample. Then we measure the credibility of 
Ho and we choose whether or not to accept our hypothesis. 


For example, let's imagine that we have several dice. All 
the dice are unbiased except one which has double the 
chances of falling on six. Unfortunately the rigged die is 
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mixed with the others and it does not bear any distinctive 
marks. We choose one die for a game night and we want to 
distinguish the biased die to be sure that the chosen die is 
well balanced. 


The die is thrown 92 times : 


31513532562433653 1344 1354244652632465436616546 
2241154636433 1655 144442414564 14316555 146362534 


For Ho we define a discrete random variable X. When the 
outcome is six the value is 1. All other outcomes are 
recorded as 0 : 


Ph P(X=0)=5/6 and P(X=1)=1/6. 
x 
x | x0=0 | xal 
Hy p | p=56 | p= 1/6 
HE PoX* PX, 
oH 1 X  o2=polxo—w) + p(x) 


The mean of X is 1=1/6~0.167 and the standard devia- 
tion is o=V5/6~0.373 . 


In our sample there are sixteen sixes and the sample mean 
is X=16/92~0.174. Therefore x-p=t,.0/1n which 
gives: 


t,.=(%—1) vn/o=0.187 . 


‘The right tail of the Gauss distribution for values greater 
than 0.187 has an area of 0.43, showing that «~43% ", 


7 Here o is known and n=92 is large enough to use the central limit 
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distributions 
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If Hy is false then H, is true. If the alternative hypothesis H, 
was "the die is biased” (with no details about the way the 
die is loaded) we would have conducted a two-tailed test 
and for @ we would have to consider the two tails of the 
distribution. This scenario only requires a one-tailed test: if 
Hy were false, the probability of observing six would be 
doubled and we would observe greater values. 


For H, we define the random variable Y with P(/Y=0)=2/3 
and P(Y=1)=1/3. The mean is t'=1/3=0.333 and the 
standard deviation is 0’=V2/3~0.471 . 


2'-0'lVn and t,,"=('-X)Vn/o'~3.24. 


Then X—w 
‘The left tail of this distribution has an area of B~0.06% . 


We can therefore very comfortably accept the hypothesis 
that the chosen die is balanced. In the case of rejection we 
would have a 43% chance of making a mistake (we try to 
minimize this error first, classically it is only below the 
threshold of 5% that one begins to question the null hy- 
pothesis). With regard to the alternative hypothesis, there is 


theorem, 


less than a 1 out of 1000 chance that we considered the die 
balanced while the die is rigged (we also talk about the 
power of the test : n=1-6). 


Note that we never calculate the probability that a 
hypothesis is true but the probability to reject the 
hypothesis while it is true (error of the first kind). 


In the legal framework an error of the first kind is made if 
an innocent person is convicted and second kind if a guilty 
person is acquitted. The jury is asked to prove the guilt 
beyond a reasonable doubt and if the person is convicted a 
must be sufficiently small [vi]. We try to minimize the 
probability to condemn an innocent person. We do not 
directly consider the probability of being guilty, the person 
on trial is presumed innocent, a defendant is considered not 
guilty as long as his or her guilt is not proven (Hy : "the 
defendant is not guilty"). 


Exercises 7 and 8 treat different cases for this test. 


H. Chi-squared test 


‘The Chi-squared test is an another hypothesis test that is 
simple to use. It tests a null hypothesis stating that the fre- 
quency distribution of events in a sample is consistent with 
a theoretical distribution. We consider different disjoint 
events that have a total probability of 1. 


number of observa- —5——G — 7 
tions : aa : 7 
expected ETE . oF 
frequency : : 


We compute the following sum : 


Next we have a table on page 178 to estimate the probabil- 
ity of rejecting the null hypothesis Ho when it is true. Ac- 
cording to the value of y%~ and the number of degrees of 
freedom we determine whether the assumption is accept- 
able. The number of degrees of freedom is : 


ddl=c—1 (number of categories minus one) 


Let us illustrate with the experiments carried out by the 
botanist Mendel. He makes crosses between plants. He 
crosses peas with pink flowers. His theory implies that he 
must obtain 25% of peas with red flowers, 25% of peas 
with white flowers and 50% of peas with pink flowers. This 
result is derived from the random encounter of gametes. 
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Imagine that he observes one thousand flowers with the fol- 
lowing values: 27% white, 24% red and 49% roses, Should 
he continue to believe in his hypothesi 


Observed numbers : 270 [| 240 | 490 | 


Theoretical frequencies : 250 | 250 | 500 | 


then 
(270—250)° af 240-250)" (490-500 
250 250 500 

and ddl=3-1=2 


According to the table, there is more than a 30% chance 
that the assumption will be rejected when it is true. 

We then decide to accept the hypothesis. In general, we 
take a critical probability a of 5%, below which it is 
envisaged to reject the hypothesis. 
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‘The test is easily generalized for a table : 


Experimental frequencie ‘Theoretical frequencies : 


JOn C2 ~ Oy ~ A) [En En ~ Ey ~ Fie 
21 Onn En E, 

Oo, oO, Ey 

2, 0,4 \Ea E,| 


The ddl is dependent on the number of columns ¢ and of 
rows r: 


ddl=(c—1)(r—1) 
We compute the %° with a similar formula : 


(0, Ei) 
E 


i 


Moreover, we use the same table to determine the validity 
of the hypothesis. 


32 


1. The sources of the uncertainties 


A random variable has a small uncertainty if the 
measurement is precise, accurate and the acquisition 
system has a good resolution. 


Accuracy is ensured by the absence of systematic errors. 
There could be a bias that makes the measurement inaccu- 
rate (even if the dispersion is low). Reading errors, absence 
of systematic control and corrections of influential factors, 
hypotheses in the modeling, etc. All biases must be identi- 
fied and estimated in order to be added to the dispersion, so 
the system becomes accurate. 


Precision pertains to the repeatability and reproducibility 
of the measurements. The values of a precise system have 
low variability. The dispersion may be caused by accidental 
errors or by a random physical phenomenon (such as ra- 
dioactivity). The experimenters by their own work, consci- 
entious and according to a well defined and rational proto 
col, can minimize dispersion. The sources can be countless, 
but we will try to identify a maximum of sources in order 
to evaluate them. 


The resolution of a sensor depends on the distance between 
the graduation marks, the type of vernier or the number of 
digits on the display screen. Sometimes other factors have 
to be added to the uncertainty due to discretization. You 
have to refer to the technical datasheet, the instruction 
guide or contact the manufacturer for a good knowledge of 
your measuring tool. Calibration of measuring instruments 


33 


can also be performed with a high-precision apparatus that 
is used as a reference. 


The influence of these different sources of uncertainty can 
be illustrated by a target and arrows. The center of the tar- 
get corresponds to the quantity to be measured and the ar- 
rows represent the different measurements. If the arrows as 
a whole are not correctly centered, the accuracy is not as- 
sured. The tightening of arrows represents precision. The 
distance between the circles on the target indicates the res- 
olution. The value noted is that of the circle whose arrow is 
closest. The experimenter sees the arrows and the circles, 
however he does not know where is the center of the target. 
He holds the bow and his desire to be closer to the center 
of the target shows the quality and rigor of his work. 


‘Measure accurate, precise and with a good resolution : 
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Measure accurate, with a poor precision and a low resolution : 


Measure precise but with a bias and a low resolution : 
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Measure biased, with low precision and resolution : 


‘The full standard deviation will be determined from the de- 
viations of each source by adding the squares (due to the 
propagation of uncertainties explained in Chapter 
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J. Exercises 


Exercise 1: Ages — Answers p168 


Students of a class have the following age: 5 
18; 19; 18; 18; 18; 17; 18; 19; 17; 19; 17; 21; 18}. De- 
termine the mode, median, arithmetic mean, geomet- 
ric mean, range, standard deviation, root mean 
square deviation, and mean deviation’. 


Exercise 2: Card game Answers p168 


We play with a 32-card standard deck. We randomly 
draw five cards. 

a) Determine the probability of a four aces hand. 

b) Determine the probability of having a flush (a flush 
is a poker hand containing five cards all of the same 
suit - hearts, spades, diamonds or clubs). 


Exercise 3 : Gravity field 
Answers (Complete Edition) 


Students measure the Earth's gravitational field 
strength g. The students measure the following 
values at the laboratory : 6,20 ; 8,35 ; 13,00 ; 8,3 
8,54 ; 9,67 ; 9,75 ; 10,66 (m/s*). 

a) What comments can be made about these results? 
b) Calculate the mean and standard deviation. 

c) What is the mean uncertainty (95% confidence)? 
Is the result consistent with the expected value? 

d) A ninth student performs a new measurement 
under the same experimental conditions. Evaluate 
the probability that the student will get a result 
between 8 and 12 m/s’. 


8 Absolute deviation mean=|> |x, 
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Exercise 4: Elevator Answers (Complete Edition) 
The maximum load of an elevator is 300 kg, and the 


total mass of the occupants is 280 + 10 kg at «. What 
is the probability of being overloaded? 


Exercise 5 : Assignment Answers (Complete Edition) 


The following table represents Students' assignment 
grades (ranking on a 20 point scale) : 


10 


i] 7] 6] 9] 5] 5] 10] a5 


2fisf afufuf s| 6] i2|ao[ie 


w]i2{a| if sfas[ai{io; 2] 7| 2{ @ 
6 


a) Calculate the mean and standard deviation. 


b) Make a graph with the grades on the x-axis and 
the frequencies on the y-axis. 


c) Make another diagram with the following class 
intervals : [0, 1, 2], [3, 4, 5]... , [18, 19, 20]. 
Which bar chart do you prefer? 


Exercise 6 : Yahtzee Answers (Complete Edition) 
We play this game with five six-sided dice. 


1) The five dice are drawn. What is the probability 
of having Yahtzee (all five dice the same). 


2) What is the probability of having a sum smaller 
than ten? 
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3) We do a series of throws and we get the following 
sums: 18, 15, 17, 22, 16, 12, 14, 22, 23, 14, 23, 14, 
18, 21, 12, 15, 18, 13, 15, 18, 17, 15, 17, 21, 25, 16, 
8, 15, 15, 13. 


a) Calculate the mean and standard deviation. 
b) What mean do you estimate with 95% 
confidence? Is it consistent with the theoretical 
value? 

c) Make a graph with the values and their 
frequencies. 

d) If we roll the dice again, what is the 
probability of the result being higher than 24? 


Exercise 7 : Elastic bands 
Answers (Complete Edition) 


An elastic bands manufacturer indicates that among 
a thousand elastics sold, on average, ten are not 
functional. A buyer wants to test the delivered 
batches before accepting the delivery. He decides to 
refuse the delivery if the number of damaged elastic 
bands is too high and wants to have a less than 1% 
chance of making a mistake by refuting the manu- 
facturer's indication. The buyer picks n elastics 
randomly. How many damaged elastics should the 
delivery contain for the client to refuse the delivery? 
Indicate this number for three different cases: a 
sample of 1000, 200 and 50 elastic bands. 


Exercise 8 : Testing an insulating panel 
Answers (Complete Edition) 


A manufacturer specifies a thermal conductivity of 
0.039 W/m/K for a insulation board. The value is 
certified within +5%. You want to check if this is 
true. To do so, you take ten panels at random and 
measure their respective conductivity (mW.m*.K") : 


ao 


39.1]38.8]39.5]39.2]38.9]39.1]39.2]41.1]38.6/39.3 


Are the values in agreement with those announced by 
the manufacturer (95% confidence on the given 
margin is consider)? Could he, according to your 
results, announce another margin? 


Exercise 9: Coins Answers (Complete Edition) 


We perform a large number of coin tosses to test if 
the probabilities of landing heads or tails are equal. 
We carry out the experiment with three different 
coins. Are they balanced? (Answers with 95% 
confidence) 


1) 42 tails and 58 heads. 


2) 510 tails and 490 heads. 
3) 420 tails and 580 heads. 


Exercise 10 : Parity Answers (Complete Edition) 


Is gender equality respected in both chambers and The 
Supreme Court? 


Male Female 
Parliament 410 107 
[Senate 272 76 
The Supreme Court 10 2 
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Exercise 11: Births Answers (Complete Edition) 


Let us test the following hypothesis : births in 
Sweden are distributed uniformly throughout the 
year. Suppose we have a random sample of 88 births. 
The results are grouped according to seasons of 
variable length : 27 births in the spring (April - June), 
20 in the summer (uly / August), 8 in the fall 
(September / October) and 33 in the winter 
(November - March). 


At a 5% critical level, can the hypothesis be rejected? 


Now, we collect a very large sample : 23385, 14978, 
14106 and 35804 


What is the conclusion? 
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Theory 


Exercise 12 : Gaussian distributions in the plane 
and the space 
Answers (Complete Edition) 


One-dimensional Gaussian distribution : 


Let p(x) be the probability density function of the 
standard normal distribution. 


1- Calculate and compare the means of x and |x|. 
Which is equivalent to the mean distance from the 
origin? What do you think about a, and oj? 


2- Do a numerical calculation of P(\x|<1), P(\x|<2) 
and P\(|x|<3). 


‘Two-dimensional Gaussian distribution : 


Let p(x,y) be a two-dimensional standard normal 
density with p(x,y)=p(x)p(y). Where p(x) and 
p(y) are one-dimensional standard normal densities. 


Hints: 


Let consider a multiple integral of a two-variable 
continuous function. If we can separate the variables 
and the limits of integration are independent of x and 
ye 

SS Flix ylaxdy=ff f(x) f(yaxdy=J f(x)de ff (y)ay 


Converting between polar and Cartesian coordinates : 
p'=x'+y* and dxdy=21p dp (rotational symmetry) 
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1- What is the expression of p(x,y) ? Show that 
p(x, ¥) satisfies the two necessary conditions for a 
probability distribution. 


2. By introducing the polar coordinates verify that the 
probability on all the plane is one. You will express 
p(p) define as: 


Sl plx,y)dedy=f p(p) dp. 


p(p) is the density function with respect to p. 
p(p) dp corresponds to the probability of an event 
being between p and p+dp. 


What is the value of the mean j of the distance p from 


the point of origin? What is the standard deviation 0, 
for this distribution? 


3-Caleulate P(p<o,), P(p<2a,) and P(p<3a,), 


Three-dimensional Gaussian distribution : 
Let p(x,y,z) be a three-dimensional standard normal 


density with p(x,y,z)=p(x) p(y) p(z). Where p(x) , 
p(y) and p(z) are one-dimensional standard normal 
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densities. 
Hints: 


Same properties for the multiple integral than in two 
dimensions. 


Converting between spherical and Cartesian 
coordinates xtty*tz? and dydydz=4mr? dr 
(spherical symmetry) 


1- What is the expression of p(x,y,z)_? Determine 
p(r) define as: fff p(x,y,z)dxdyde=f p(r)dr. 

p(r) is the density function with respect to r. 

p(r)dr corresponds to the probability of an event 
being between rand r+dr. 


2- Verify that the probability on all the space is one. 
3- What is the value of the mean F and the standard 
deviation a, ? 

4-Calculate P(R<o,), P(R<2o,) and P(R<3o,). 


5- Compare the three cases. 


Gaussian 1D 2D 3D 
Distance from 9 
the origin kl p i 
2 
Mean \2 
Standard 1 v2 
deviations 
P@) 68.3% 60.8% 
P Qo) 95.4% 98.2% 99.3% 
P Go) 99.1% 99.98% | 99.9994% 


‘To check your calculations on a spreadsheet you can use the 
following functions: 


On OpenOffice 
+ sum of a selected area * (ex: B43:B53) 
=SOMME(*) 


+ value set to the cell B3 : $B$3 

+ =MOYENNE(*) 

+ squared value : #*2 

+ square root : **(1/2) 

+ =ECARTYPE(*) 

+ Student's t-value 95% confidence and 
OLSTUDENTINVERSE(0,05;19) 
'EST.KHIDEUX(*;*); * : experimental frequencies 
(ex.: B71:E72), ** : theoretical frequencies. 


=20 


‘9 war: We may be surprised at the difference of the one-dimensional 
expression with the absolute value, it is only a question of definitions in 
cylindrical and spherical coordinates. For example, pe(0:+so{ and 0¢[0:2a1 
but we could also take pe)-co:too[ and Be[O:nL. then the mean of p would be 
zero and we would have considered its absolute Value 


45 


IL. CORRELATION AND 
INDEPENDENCE 


In the previous chapter we looked at a singular ran- 
dom variable X with 1 number of outcomes (x)}. Now we 
have several random quantities and a new index to distin- 
guish them : X; and its observations {x}. X) is the j"" quan- 
tity and x» is the &'" observation of this quantity. We are in- 
terested in the interactions between these different quanti- 
ties. 


To illustrate, we consider a sample of four individuals 
with three characteristics, of height X,, weight X2 and of 
birth month Xs. A priori, we expect a correlation between 
height and weight : generally taller people also have heavier 
body mass (positive correlation). On the other hand, we can 
think that birth months have no effect on weight and height 
(X; uncorrelated with X, and X2). 


A. Correlation coefficient 


‘The sample correlation coefficient r is used to identify a 
linear relationship between two variables X, and X; : 
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r varies between -/ and +/. If |r the variables are 
perfectly correlated : r=1 is verified in the case of a 
perfect increasing linear relationship, and. r=—1 in the 
case of a pel inear correlation. If r=0, 
the variables are uncorrelated and independents. 


Calculus of rip ,rys and ry 2 


X: | X: [x ee ae 
(em) | (kg) 
1 fieo}e¢[a] -1s | -1 2 225 
2 [170] 66] 8] -5 EY 2 25 
3 [1sofsa]o] 5 9 3 25 
4 [190] 86/3] 15 ul 225 
x[175| 75 [6 500 
and: 
(8!) Oa-mP] One (K-, 7 
(x= (xy-% (xy-% 
121 4 165 30 22 
81 4 45 -10 -18 
81 9 45 15 27 
121 9 165 -45 
404 26 420 -10 
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ry=—0.02 


rz is close to +1, so we have a strong positive correlation. 
riz and rs are close to zero and therefore: X; is independent 


of X; and X> 


0 


160 170 


xt 


190 


x 


Examples of data sets : 


Example 1 


x2 


0 
. 
a oe 
ye 
. 7 
. . 
2 
150 1 180 6 70 0 90 
xt x3 
Example 2:1-08 
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Example 5 Example 6 :1=1 
e2 2 
a 24 
y * qquanccsaneneye® 
2 2 
8 
3 t 15 9 2 
xt x1 


. 
5 
poset’ 
= 
Example 9: =0 Example 10 :'=0.94 
cosas 
oS 
oa 
2 6 
4 
102 46 
0 


Examples 7, 9 and 10 illustrate a strong. correlation 
between two variables. Yet the correlation coefficient is not 
as close to -1 or +1 as we could imagine, it is even zero in 
example 9. This is due to the fact that the correlations are 
not linear. 


There may be saturation phenomena (example 10) or 
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threshold effect (a product may be beneficial at low dose 
and harmful at higher doses - example 9). To prevent these 
phenomena from misrepresenting the data, it is important 
to carefully consider the relevance of the variables chosen 
before starting a statistical study. 

Another example: if we study the volume V of different 
objects according to their size 7, we find a positive 
correlation. But the correlation will be much stronger 
between Vand X = T? (graphs 7 and 8). 


The fact that two variables are correlated does not 
necessarily imply a causal relationship between the two 
variables (the variation of one variable leads to the 
variation of the other). Rather than the variables effecting 
each other, the change may be attributable to a common 
external cause. 

For example, it can be assumed that there is a correlation 
between the consumption of sunscreen and of ice cream. 
There is obviously no causal link between the two but a 
common cause i.e. the weather. 

‘A physics study can show a causality, not a statistical one. 
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B. Propagation of uncertainties formula 


Consider a bucket filled with a million grains of sand. 
‘The mass of a grain is 10 mg with an uncertainty of Img. 
‘Whaat is the mass of sand contained in the bucket? 


1) Propagation of standard deviations formula 


‘Asa general approach, let / be a function of p independent 
variables: 


[(%4,%, 


Each of these random variables is associated a mean value 


x) and astandard deviation, . 


What are the values of f and o, ? 


Statistics give the answer and demonstrates the propagation 
formula of the standard deviations: 


10 We obtained the variance formula by replacing o* by V. 
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2) Uncertainty calculations 


For the uncertainties (defined on page 13) we also have a 
propagation formula: 


The uncertainty propagation formula is not as exact as for 
standard deviations, but this formula is very practical and 
often very close to the exact result. 


Considering our bucket: —-M (m,,my,...,M),...,m,) 
F 
with M=) m, 
A 


where we call M the total mass of sand in the bucket, m, 
the mass of each grain and p the number of grains. 


6 M16 m,'A m2 


@M/ém,=6m,/Om,+..+0m/dm,+...+m,/dm, 
OM/ém,=0+...+1+...+0= 


(Calculations of partial derivatives are explained on page 164) 
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> 
then AM {= fam’ 
a 


and AM*=p-Am* with Am=Am, whatever j. 


‘p-Am=y1000000%0.001 g . 


Finally : 


‘The bucket weighs 10 kilograms with an accuracy to the 
gram. The precision on the mass of the bucket is thus 
0.01%. Naively, we might have thought that the overall 
uncertainty on the bucket mass was the sum of the 
uncertainties of each grain. We would then have an 
absolute uncertainty of 1 kilogram and a relative of 10%, 
which is very different from reality and would ignore the 
compensations. 


Here the propagation formula is very precise because we 

have a very large number of grains. It is even exact, from 

the small numbers, if the distribution of the mass of the 

grains is Gaussian", 

cari : A linear combination of Gaussian quantities is itself Gaussian 
(applies here to a sum). And in the propagation of uncertainties 
formula, if f and the x have the same kind of probability 


distribution, the formula is exact like this one with the standard 
deviations, 
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In practice, there are multiple common ways to calculate 
uncertainties’ propagation, depending on the situation, 
given below: 


+ For sums or differences we add absolute 


uncertainties squared: 
AP=DA 
a 


v with Av=Avx/=/em then 


For example if d=. 
Ad~=14cm, 


+ For products or quotients we add relative 
uncertainties squared: 


(4-2 


For example if R=U/I with U and I with a precision of 


1% then R is known with a precision of 1.4%. 


In more complex cases, the partial derivative calculation 


must be performed explicitly. 


Alternatively, using a random number generator or 


ss 


uncorrelated packets approximately Gaussian, it is possible 
to do a numerical calculation. 

The latter method is illustrated in Exercise 2 of this 
chapter. A spreadsheet can do the calculations auto- 
matically (for example see on sheet 4 of the file 
IncertitudesLibres on www. incertitudes.fr). 


There are also methods which give general ideas on 
uncertainty. They develop a general impression at the risk 
of reliability. 


For example as we add the uncertainties squared, we can 
anticipate that the largest uncertainty will quickly prevail 
over the others, Consider the example of R=U/, if U is 
known with an uncertainty of 1% and / of 0,1% then R is 
known with an uncertainty of 1,005%~1%, we can ignore 
the uncertainty of 1. 


For addition and subtraction, it is sometimes considered 


that the parameter with the last significant figure the less 
precise indicates the precision of the last significant figure 
of the result. 

Yet on our example calculation of the mass of the sand- 
filled bucket, it does not work. Since the mass of a grain is 
m = 10 mg but the mass of the bucket M is known to the 
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gram and not to the milligram! 


For multiplication and division, it is sometimes considered 
that the parameter with the lowest number of significant 
figures indicates the result's number of significant digits, 
but here too one must be careful. 

‘Asa simple illustration, if H=2h with h=5,00m (h known 
within a cm), what is the value of H? According to the rule 
below, H would be known with three significant digits: 
H=10,0m. H would only be known to 10cm, it goes without 
saying that it is closer the cm ... 


While these tricks serve to aid the calculations, they are not 
without their pitfalls and must used with caution. 
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C. Linear regression 


We have two correlated random variables, X and Y, 
and are in need of an approach for modeling the relation- 
ship between them, The outcomes generate a cloud of data 
in the plane y(x) and we want to determine the best affine 
function y=ax+b that fit the observations. For example, 
what is the most appropriate relationship between the 
height X and the weight Y in our initial example? 


What are the uncertainties Aa and Ab? 
1) Principle and formulas 
Crs 


yl varies 


b 
varies. 


We choose the least squares method: this method mini- 
mizes the sum of squared residuals. 


The set of dots is denoted M,(x,,y,) . For xi given, the es- 
timated value of yis: J,=ax,+b. 
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We have to minimize the following quantity: 
LHL (w-H! 


We differentiate this quantity with respect to a and b, and 
we set the derivatives equal to 0. We then have the best 
straight line and can obtain the following equations: 


Yly-ax-b)x,=0 and DY (yax—b)=0. 


This is equivalent to 


a = . 
for the residuals 72 


+ for the slope 


+ for the y-intereept 


12. Demonstrations p97. 
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Then [Aa=t, Ab=i, 


2 : Student's t-value for n-2 degrees of freedom. 


You are now able to carry out all calculations. 
Let's do the calculations for weight in relation to height: 


160x64-+170x66-+180x84 + 190x86)/4 


(160°+170°+ 180°+190")/4 


a=(13230—175 x75)/(30750—175° 
b=75-0.84 x175=—72 


s,=V|(64—(0.84 x 160—72)}'+(—4.8) +4.8'+(—1.6) ]/ 


06 /V(160—175) +(—5)+5°+15 ~0.226 


Aa~2.92 x0.22 


.66 with a 90% confidence 


5425.06 V(160"+170 Wa (as +574 254225) 
Ab=2.92x39.7~116 with a 90% confidence 


0.84+ 0.66) Weight —(72+ 116) 


then: Height= 
with a 90% confidence. 


Here, the formula is very imprecise, which is unsurprising 
given the small number of points and the dispersion of the 
data, However, the method of calculation is now explicit 
and comprehensible. 


In the graph that follows we have: 
+ In the middle, the interpolated straight line repre- 


sents the best balance between the points above and 
below this line. 
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Cn en ee ee rn 
Heght x 


The dotted lines represent the two extreme lines ( 
YM X FB, ANG Y= Oy X+B yay) 

‘The first curves represent the estimated values for y. 

It is the mean confidence interval of y, correspond 

ing to x: 


For example if x,=/75 em we can calculate 
Yu=75.0£7.4 kg, Also we can obtain an estimation 
out of the interval, for example if x,=/95 cm we 
obtain y,=92/5 kg. 
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+ The outer curves represent a prediction for a new 
‘measurement. Prediction interval for an observation 
Yo: 


For example, if the height equals 175 em there is a 
90% chance of their mass being between 58 and 92 
kg (generally 90% of the data are inside the curves 
and 10% outside). 


2) Absolute zero measurement 


We study a gas en- 
closed in a rigid container 
of constant volume. We 
have sensors to measure 
its temperature and pres- 
sure, Initially the gas is at 
ambient temperature and 
pressure. Then we im- 
merse the whole container 
in hot water and measure 
the changes over time! : 


1 This experiment was realized on Tuesday 17 October 2006 
in Bourges by M. ROUAUD and 0. LEROY at the Lycée Alain- 
Fournier (France). 
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time 10h [10h [10h |10h not 
15 [30 [40 |55__|noted 
temperature | 19.8] 52.9] 47.8] 42.4] 36.2 
ja °C) 
pressure 1013] 1130] 1112] 1093] 1072] 1061] 1049 
P (hPa) 


We assume that the gas obeys the ideal gas law 
PV=nRT=nR(O-,,). Plotting O(P) we can obtain 


a temperature of absolute zero : the y-intercept give Oox. 


Pressure hPa 


The regression is good (r=0.99991) but the measured 
points are far from absolute zero. By extension we obtain 
with a 95% confidence: 


Oy =— 266.02 4.8°C 
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We know that absolute zero is -273.15 °C, therefore this is 
not consistent. We can therefore assume that there is a bias 
and that we have not considered all the sources of 
uncertainties. 


3) Regression with uncertainties on the data 


‘The measuring instruments are not perfect and the 
manuals indicate the precision for each. The uncertainties 
are 1% for the pressure sensor and 1°C for the 
thermometer. We thus have an uncertainty on x and y : 


M(x,tAx,,ytAy;) 


Now both the dependent and independent variables 
have uncertainties which contribute to the weighting 
factors. Let w: be the weights. In an experimental situation 
the weights are frequently not equal. The linear least- 
squared fit requires the minimization of : 


DXwe? — with 


The problem is solved iteratively because the weights 
depend on the slope. We initially put an estimated value of 
a, then the value of a obtained replaces it until the equality 
of the values. 
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We obtain: @,,=—266+35°C with the same con- 


fidence on the uncertainties of x, and y. The value is now 
correct. The main sources of uncertainties seem to be 
included, 


We could also consider the modeling uncertainty induced 
by the ideal gas hypothesis, but under the experimental 
conditions of this experiment, the model provides a good 
approximation. This source of uncertainty is negligible 
compared to the others uncertainties considered here. The 
use of a real gas model (like Van der Waals equation of 
state) would demonstrate this. 
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Formulas [i] : 


8=> wly-(ax,+b)P 


with 


A= Dw Dowd = (Smal 


then 
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4) Linearization 


In many cases we can go back to a linear model (i.e. 
ax+b ). Here are some examples below: 


yi=ax'+b with 
y'=In(y) 


yis= 


(logistic 
distri- 
bution) 


» (Pareto 
distri- 
bution) 


\x] 


y'=In{In 


x'=In(x) 


no linear model 


no linear model 


yau+Bxtyx no linear model as y(x) 


67 


5) Comparison of methods 


In all the regression methods we consider, we are interested 
in the variability of y to x fixed (the opposite approach 
would yield equivalent results). 


For a given x, we have a y; and its standard deviation oy. 


a) Summary 


1- Simple linear regression 


y=83 ysl73 y 37.3 yr=38.3 
s=0443 and 51.98 


Case}: a=5 b=5 5,=2.34 


Simple regression does not mean that the data has no un- 
certainties. The uncertainties are unknown and we estimate 
them with the data itself. The uncertainties are considered 
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constant whatever regardless of y,, The uncertainty corre- 
sponds to the standard deviation of y, with respect to the es 
timated line : 


2- Regression with constant standard deviation sy: 


55 5,=0.482 — 5=2.16 


In this case the sy are equal and known: s,=s, and 


s x 


69 


If the straight line does not pass through the error bars, it 
can be assumed that all sources of uncertainty have not 
been calculated. It is necessary either to integrate them into 
5,, or to apply the previous method. 


3+ Regression with standard deviation s,: 


S553 swH25 sy=2 sel seal 
Case 3: a=4.67  b=6.51 0.482 54=2.16 


sud 


The sy are known. We can apply the propagation formula 
of the standard deviations: 


z (3 


‘The formulas’ results are exact and help us to find the ex- 
pressions of the previous cases. Also in this case we can 
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use the following estimates: 


Aa= 


Ab= 


1 
mM 


4- Regression with standard deviation s, and s, 


Everything happens as if only y, had standard deviations s. 


a 


Hence the formulas : 
y (22 ‘ 
7 \Oy: 
The derivatives are difficult to calculate (the weights de- 


pend on a), but we can easily evaluate them numerically. 
‘Also we commonly use the following estimates: 


Case5:a=5 b=5 5=0,696 5% 
Uncertainties in this case can come from measuring instru- 


ments. The dispersion of the first case on page 68 can come 
from accidental errors linked to the experiment or from a 


R 


fundamentally random phenomenon. 


We would like that the fourth case on page 71 includes all 
the sources of uncertainty of the cases 1 and 5 : 


1.443 and S,,~0.696 but s,,~0.695 


Using the conventional formulas, we get the impression that 
the dispersion around the regression line is not included. In 
order to get the correct dispersion around the regression 
line, we perform the direct calculation with the propagation 
formula and I propose the small variations method: 


Where yi. are kept constant; y+ y,>a+Aa 


Ayy is a small variation with respect of y; , if Ay, becomes 


smaller, _ stays constant (definition of the derivative). 


We try to find the result for s,5: we replace y=0 with 
y=10.001 and then from a=5 we have after iteration 


.999907 then 22 0.093. 


or 


We return yi= /0 and repeat the procedure for y2 by re- 
placing it with 15.00/. We obtain the following results: 


B 


j=2 | i=3 ] d=4 | i=3 | i0 | J=7 


-0.078 |-0.049 |-0.006 }0.041 0.080. Jo.105 


We find s,~0.696, the same result than the previous 


method. 


Let's do the same for Su«: 


j=2 | i=3 | d=4 | i=3 | i0 | J=7 


-0.080 |-0.050 |-0.007 Jo.081 0.122 Jo.147 


We then find that s,~0.786. The result is significantly 


different from the classical estimate and now seems to in- 
corporate all the sources of uncertainty. 


In the exercise Standard deviation proportional to y on page 
98 we study a particular case where we carry out the direct 
analytical calculation. The comparison can thus be ex- 
tended. 
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D. Nontinear regression 


We generalize the weighted least squares method in the 
nonlinear case. The function is nonlinear with respect to x 
and also nonlinear with respect to the parameters. Although 
multiple regression has similar developments, it is not dealt 
with in this book [x]. 


1) Principle 
y 


We compare y; to the value f(x,) given by the function 


sought: ¥i—f (Xi). The weight assigned to (Yi—f (xi)? 
is inversely proportional to the variance of y,—f(x,) . 


‘The quantities x, and y, are independent’ from where: 


13 Talking about independence between two variables as we look for a 
functional relationship between them may seem illogical. We refer 
here to the experimental determination of each quantity which is, 
independent (in the sense of uncorrelated uncertainties) 
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V(y- FX =V(y)FV( F(x) 
By applying the variance propagation formula again: 


V( F(x =F (xP V(x) 


into S 


Dwly- fl? 


with w= 


+ f(x) o,, 


allows to determine the parameters a, of our 


function (by an analytical computation or a numerical 
resolution). 

Each time we can return to a system of linear equations of 
the form HA=B, with H a square matrix and A the vector 
associated with the parameters. Then A=H"B, where H” is 
the inverse matrix of H. 


‘The parameters uncertainties are given by the diagonal 
terms of the matrix H” : 
o=(H'),o7 where o,° is the residual variance with 


respect to the estimated curve. 


When there are no uncertainties on the data, the standard 
deviation of the residuals with p parameters are written: 


148? is also called 77. If we assume the distributions Gaussian, the 
standard deviations can be replaced by the uncertainties using 
Student's t-values. 


16 


np 

When 1; depends on the parameters we iterate the method 
until we can consider the weights to be constant. If we 
know the standard deviations of the data, the standard 
deviations of the parameters can be computed with the 
propagation formula, or they can be estimated using the 
same procedure used for linear regression with error bars 
on page 66. 


2) Polynomial regression 


In that case: 


f(x)=a,4a, x-+a,x°+.. 


This function is non-linear with respect to x and linear with 
respect to the parameters. 


Let us illustrate with the example of Cauchy's equation. 
This equation explains the phenomenon of light dispersion. 
It is an empirical relationship between the refractive index 
and wavelength: 


with the following data: 


(um) 0.6157 | 0.5892 | 0.5685 | 05152 | 0.4981 
In 1.71276 | 1.71578 | 1.71852 | 1.72716 | 1.73060 
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‘The uncertainties on 4 and n are initially neglected. What 
are the values and the uncertainties of ao, a) and a2? 


‘We have the following parabolic regression: 


+ x=t/N. 


f(x)=ata.xea,x with f= 


SSD (y-ay—a,x,-a)x}) and 


‘we obtain: 


then HA=B with: 


With a spreadsheet we can easily do the calculations: 


133 1 | 4150 —2530 376 
H=33 11 38| + H =|-2530 1546 —230) , 
[41 38 135, 376-230 34,3 
1.7\ 1,68129| 
5.7| then A=H*B=|0.01135] . 

\19 {0.00022 


For the uncertainties: 


JutooSe with 
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-87x10° and with 95% confidence t,.;=4.30 
then 

ay=1.681340.0017 , — ay=(1.13£0.10)x10° wm* 

and a)=(2.2+/.5)x10% um! 


Now, if we take the uncertainty An=0.00004 with 95% 
confidence, we have an uncertainty on y but not on x. 
Therefore the weights are constant: w,=w=1/An*. We 
obtain the same system of equations and we have the same 
results for H, 1 Band A. 

Here, in a similar way to the linear regression with error 
bars, we can estimate the dispersion of the residues by 


an= |G then 


= 1.6813£0.0012 , as=(1.140.07)x10" um? 
and ay=(2.2+1.0)x10" uni? 


With all the uncertainties, An=0.00004, Ai=0.005.m and 
Ax=2AA/', the weights depend on the observations. We 
nevertheless arrive at the same system of equations by 
considering the weights on an iteration constant, and we 
compute H, H", B, and A with estimated parameters. 

To calculate the means, we use the expression of the 
following weight: 


1 


Wi) 8 oe ee 
An’+(a,+2a,x;) Ax, 
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We thus obtain a first expression of the parameters. To 
iterate, we replace the previously used estimated 
parameters with these new parameters. We iterate as much 
as necessary to obtain a consistent value. Convergence is 
often very rapid: 


Iteration a ay ay 


Estimated [1.5 0.005 (0.0001 
1 |1.681282848208 ]0.011350379724 [0.000219632215 


2 |1.681269875466 ]0.011358254795 |0.000218466771 


Hy 
Vow, 


ao=1.6813+40.0013 ,  ay=(1.14£0.08)x10" wm* 
and 241.2)x10% um* 


Also here we consider: A a, then 


3) Nonlinear regression 


We will start from p parameters a: estimated and use an 
iteration method. On each iteration, the weights will be 
considered constant and the function will be linearized for 
each of n points on the set of parameters. 

‘The function depends on the x, and the parameters a. 


Then we have f,;=f(x,;a,) - 


80 


a8 _ ot OF i ig 
Bap 2 Gq, YAI=0 


8=Dwly-f)? and 


The first estimated parameters are noted a s. Following 
parameters will be noted a), for the jth iteration. We will 


carry out a linearization with a small variation dao, around 
is 
aon. 


Let a=(a, 


from where again HA=B and A=H"B. We iterate until 


15, art: We generalize the notion of derivative by adding the 


variations according to all the parameters: 
PUxote)= Flx9)Fe( F(X), 
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variation on the parameters is negligible and the values 
converge. 
Here also, we use: 


H"\wo?, 


Let us illustrate with a biological experiment, in which the 
relation between the concentration of the substrate [S] and 
the reaction rate in an enzymatic reaction is studied from 
data reported in the following table [ix] : 


i{l 2 3 4 5 6 7 
[s] 0.038 Jo.194 0.425 [0.626 [1.253 [2.500 
v [0.050 {0.127 [0.094 [0.2122 |0.2729 |0.2665, 


‘The model can be written as: 


=| and a={2|, 
\8] \B.| 


In the absence of given uncertainties on x, and y; : 


82 


% XP 


7 (Bo+X,)" 


and 


— fai) B=-y— 
(yi fo.) and Bex) 


with fo 


produces: 


0.649 0.508) 
\0.508 0.668 


From =a) +5 and B)=Bo+dBo we have the new 
estimated parameters: 


,=0.9-0,567 =| 


.333 and B,=0.2+0.06~0.260 


We repeat the calculation from the start, this time using 
these values instead of a and fy to iterate. We calculate 
new matrices and vectors H, H’, B and A, and obtain a2 


83 


and p>. 
‘The results are shown in the following table: 


tera |g B ba op 


0.9 02 |-0.57 — |0.060 | 1.4454965815 


1 [0.333 0.26 0.0101 |0.166 —_|0.0150720753 


2 |o343 0.43 Jo.o1so o.103 — Jo.o084s83228 
3 10.358 0.53 0.0040, 10.024 0.0078643240 
4 10.3614 0.0004 10.0024 0.007844 1826 
5 {0.36180 0.00003 |0.00018 }0.0078440067 
6 {0.36183 0.000002 0.000013 |0.0078440057 
i é {1-52 6. el 
After enough iterations: igus 651 


Calculate the uncertainties on the parameters: 


Aa=\(H 


then Aa=0.07, AP=0.35 . 


Eventually: @=0.36%0.07 and f=0.56:40.35 
with 80% confidence. 
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The following graph shows the reaction rate as a function of substrate 
concentration, The squares are the experimental points, the solid line is 
the optimal curve and the dotted lines are the two extreme curves 


Pop. and fap. 
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‘To consider the uncertainties on the data we need to add 
weights. These are considered constant on each iteration. It 
will be necessary to calculate the derivative of f with 
respect to x which is present in the expression of the 
weight. 

For standard deviations on the set of data we can calculate 
the standard deviations on the parameters using methods 
described on page 68 for the linear regression. 
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E. Exercises 


Exercise 1: Correlations Answers p168 


1. We carry out nine experiments to obtain in each case 
three realizations of the quantities X:, X2 and Xs: 


; 7[3 4,5 ]°]7]8]9 
a at[afofofofilfili 
‘a ol[t}ifo}alilola 

G ofifafof[i][afo[t 


a) Determine the arithmetic means and standard 
deviations of these three quantities from these 
samples. 


b) Plot X; as a function of X;. The same for X;(X:) and 
Xs(%). 


c) Calculate the correlation coefficients riz, rs and res. 
What comments can be we made from the results? 


2- Same for the following data: 


© [otTifal2z[of[alvo[2 [1t 
» [rt2fale2fof2z{[saf2 {2 


3- Same for the following data: 


a[2]2],[°0,/2]2]1 
afo,[2z2[2f[o0f[z2z [4 
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Exercise 2: Volumes Answers p168 


We fill four beakers with 100 mL of water each with a 
pipette. To test the pipette and know the precise 
quantity of water, we weigh to the nearest decigram 
and obtain the following results for the different 
beakers in mL: 


V:={100.1, 100.0 , 99.9 , 100.0} 


1- Calculate the mean and the standard deviation of 
Vi. Estimate the precision of the pipette with a 95% 
confidence. 


We now fill two beakers and gather the contents of the 
two into one: 

V=Vi+ V2. 
In the same order as Vi, we obtain the following 
measurements for V2: 


V2={100.0, 100.1, 100.0 , 99.9} 


For example, for the third measurement, V,=99.9 mL 
and V2=100.0 mL. 


2- Show that V; and V2 are independent quantities. 


3- Calculate the mean of \V, its standard deviation and 
its uncertainty AV with 95% confidence. 


4- Could you find this result with the uncertainty 
propagation formula? 


(To improve the test it would take more measurements, but 
the principle remains the same, and the results remain valid 
because we have used the Student, considered decorrelated 
data and globally Gaussian packages. We should also take 
into account the uncertainties on the measures - resolution - 
in addition to their dispersion.) 
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Exercise 3: Trees Answers (Complete Edition) 


We want to measure the distance d between two trees. 
For this we have a stick of length one meter. From one 
tree to the other, we place the stick end to end a 
hundred times. For each displacement, we estimate an 
uncertainty of 1 cm. 


What is the uncertainty estimated on the value of d? 


Exercise 4 : The two-position method 
Answers (Complete Edition) 


We measure the focal length f of a convex lens using 
the two-position method (also called Bessel method or 
displacement method). 

An illuminated object is set up in front of a lens and a 
focused image forms on a screen. The distance D 
between the object and the screen is fixed. When 
D>4f,, there are two lens positions where the image is 
sharp. The distance between these two positions is 
denoted d. We then have the focal length of the lens by 
the relation f=(D’-d?/4D. We measure D=2000+10 
mm and d=536+20 mm. 


What is the uncertainty on f ? 


Exercise 5 : Refractive index 
Answers (Complete Edition) 


We want to measure the index of refraction nz of a 
window glass. We perform the experiment of the 
refraction of a laser beam. According to Snell's law of 
refraction n,.sin(i:)=n,.sin(i,), where n, are the indices 
of the materials and i, the angles of incidence and 
refraction. We get mi=na=1, i:=30#1° and i,=20+2°. 


Determine n; with its uncertainty. 
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Exercise 6 : Cauchy's equation 
Answers (Complete Edition) 


We want to measure the variation of the light 
index n as a function of the wavelength 4 in a 
transparent material (phenomenon of dispersion). To 
carry out the experiment we need a prism, a sodium 
lamp and a goniometer. According the theory, in the 
visible spectrum, the index variation n(A) follows 
Cauchy's equation: 


n(y)=a+ 2 
z 


The sodium line spectrum is known, For each line of 
wavelength 4,, the corresponding index n is calculated 
using the formula of the prism: 


sin 


D,, is the minimal deviation angle. A=60° is the 
internal angle of the prism. These two angles are 
measured within 2' (1'=arc-minute and 1°=60’). 


We obtain the following values: 


a(nm) | 615.7 | 589.2 568.5 5152 | 4981 
Color | red__| yellow | green-yellow | green _| blue-green 
1D. _[s749.s'] sso | sse28' [s9265'] sos 

n 1.71276 | 1.71568 | 1.71852 | 1.72716 | 1.73060 


1- Determine the uncertainty for n (An is assumed 
constant). 
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2- Using Cauchy's equation find A, B and their 
respective uncertainties. 
What is the value of the regression coefficient r? 


3- We hypothesize that plotting n as a function of 1/A 
or 1/A°, will produce a better alignment of the points. 
We want to verify that the variation in 1/A is indeed 
the best of the polynomial relations. For this we take 
the form: 

n(\)=A+B.d" 


Propose a method for determining A, B and a. 
We can verify the model because we have a with its 
uncertainty. 


Exercise 7 : Wall Answers (Complete Edition) 


There is a wall with area S=72 m?, The outside 
temperature is 6°C and the internal temperature is 
maintained at 18°C. This wall is 50 cm thick and 
consists of e,=40 cm of compressed straw (thermal 
conductivity 4)=45 mW/K/m) and e.=10 cm of coating 
(4.=200 mW/K/m). The values for 4, thicknesses, and 
the temperature are rounded to within 10%, the 
nearest cm and the nearest half degree respectively. 


1- Determine the thermal resistance with its 
uncertainty of the straw for this wall (R.A.S=e) 


2- Repeat for the coating. 


3- Taking into account that thermal resistances 
associate like electrical resistors in series, determine 
the total thermal resistance of the wall with its 
uncertainty. 


4- What should be the minimum heating power of the 


house to compensate for the losses by the walls? 
(AT=R.0) 
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Exercise 8 : Insulation and inertia 
Answers (Complete Edition) 


In a low-energy house, the thermal resistances 
e/k are measured per square meter. The thermal 
resistances are 8 m*.K/W for the roof, 4 m?.K/W for 
walls and floor, and 1 m?.K/W for door and window 
frames. The R-values are known to the nearest 10%, 
The house area for the floor, the roof, the walls and the 
frames are 36 m?, 54 m?, 82 m? and 8 m? respectively. 


1- The equivalent resistances of the roof, the walls, the 
floor and frames are in parallel. Determine the total 
thermal resistance (in K/W) with its uncertainty. 


Outdoor and indoor temperatures are constant. 
2- What should be the minimum heating power of the 


house to keep the indoor temperature constant while 
offsetting the losses? 


3- We switch off the heating and measure the 
temperature over time to obtain the following results: 


{tin hours o} 1] 2] 4] 5] 6} 8} 9} 10 
T in °C 18] 16) 14) 12) 11] 10} 9} 9] 8 


Explain why the decay cannot be linear. We consider 
an exponential decay: T(t)=a.exp(-t/t)+b . Calculate b, 
a and +, with their uncertainties. 


4- The house is insulated from the outside. The lost 
heat flux corresponds to a decrease in the energy 
stored in the house. This inertia is due to the thermal 
capacity C of the materials (J/K). 


a) By reasoning on an infinitesimal time interval dt, 
find the differential equation verified by T(t) and the 
expression of question 3. 
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b) What is the relationship between t, R and C? 
Determine C and its uncertainty. 


In question 3 we could also take into account measurement 
uncertainties: time can be considered as perfectly known 
and temperature is measured using a thermometer with 
graduations all the Celsius degrees. 

For simplicity we considered that the outside temperature 
remains constant (to account for day/night variations we 
would consider sinusoidal variations and a harmonic 
approach). 


Exercise 9: Yield Answers (Complete Edition) 


Specific quantities of fertilizer are spread on 
fields and we obtain the following yields: 


Fertilizer | 100] 200] 300] 400] 500] 600] 700 
(kg/ha) 
Yield ai{ 44] 53] 63) 66] 65] 78 
(Quintal/ha) 


1- Determine the regression line that passes through 
the scatterplot. Slope, intercept and uncertainties with 
a confidence of 95%. 

2. For 550 kg/ha of fertilizer, estimate the yield. 

3- Repeat the calculation without fertilizer. 

4. If a farmer spreads 250 kg/ha of fertilizer, what is 


the probability that he will get 40 to 60 quintals of 
grain? 
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Exercise 10 : Study of a battery 
Answers (Complete Edition) 


To determine the open-circuit voltage E and the 
internal resistor r, we measure for the battery 
different values of U and 1 with a voltmeter and an 
ammeter (U = E- rl): 


range ‘unit -V 

for U accuracy 0.05% 0.003 

um (4 ]4]4]4])4]4]a]a]a]a 
731 | 731 | 730 | 728 | 724 | 724 | 722 | 721 | 719 | 716 

range unit HA ‘unit: mA 

for I accuracy accuracy #0.2% £0.0003 
40.2% £0.03 

T 2 ys fiz] oe [oyololojoyo 


a3 | 45 | 65. [2352 | 4686 | 5200] sada | e561 | 7750 | 9264 


1- Without error bars: determine E+AE and r+Ar. 


2- Repeat including the uncertainties on U and I 
indicated in the instructions of the multimeter 
manufacturer. 


Exercise 11: Thin lens formula 
Answers (Complete Edition) 


We want to measure the focal length f of a 
converging lens. At point O, the lens forms a sharp 
image at A' of an object at A. We measure OA, OA’ and 
their uncertainties (are included all the sources of 
uncertainty: geometry, focusing and modeling). We 
consider that the lens verifies the thin lens formula: 


1/0A' + 1/0A = 1/f, 


Determine f using a linear regression. Table on the 
next page. 
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[Experimental Data (mm) 
joa [AOA OA" [AOA 
35) | ___160] 5] 
530) B[___160| rr 
496) S| ___i6d | 
740) s[__a72| Fe 
350) 3s] 191 20) 
280) s|__2id 25| 
210) s]__292/ 2a| 
150) sl 730, 102) 
Theory 
Exercise 12: Answers (Complete Edition) 


For simple regression, show that: 


Exercise 13: Answers (Complete Edition) 


For simple regression, show that we can also write: 


Exercise 14: Answers (Complete Edition) 


For simple regression, show that: Ab=\TAa 
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Exercise 15 : Asymptotes Answers (Complete Edition) 


For the simple regression, show that, above and below 
the regression line, extreme line, confidence and 
prediction curves, have the same asymptote when x, 
becomes large. 


Exercise 16 : Confidence Interval and Prediction 
Interval for Linear Regression with error bars 
Answers (Complete Edition) 


Regression with error bars 
1- Give the expressions of X and © using the w, 
weights. Could you find a new expression of A with X 
and x? (p66) ? 

Analogy 

2- From the confidence and prediction intervals for 
simple regression (p61 and following), use an analogy 
to determine the following formulas: 


Confidence: Prediction: 


Ay, u 


Ye cba Ay.= 


3- Determine the y-distances to the regression line for 
extreme lines, confidence curves, and prediction 
curves when x, becomes large. 

In a previous exercise we showed that for the linear 
regression, the asymptotes are the same. By analogy 
what should we set so that it is the same in regression 
with bars of errors? 
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Show that we then have the following formulas: 
Confidence: Predictioi 


(x=) 


1 
Ay,= 2+. 
TS wi \ 


The formulas obtained by analogy are thus empirical. 
Yet, while they seem experimentally coherent, they 
require confirmation by theoretical demonstration. 


-¥ 


Confidence and prediction curves for the absolute zero 
experiment: T (°C) as a function of P (hPa) 


| 
| 
| 
| 


Exercise 17: Other expressions 
Answers (Complete Edition) 


For regression with error bars give the expression of 


a, b, Aa and Ab as a function of x, y, xy, @ and y*. 
Compare with simple regression. 
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Exercise 18 : Least squares method 
Answers (Complete Edition) 


Demonstrate by the least squares method the 
expressions of a and b: 


1- For simple linear regression. 


2- For linear regression with error bars. The Ax, and 
Ay, are considered small with respect to x, and y). 


Proof of the expressions of Aa and Ab for the simple 
regression: 


Method 1 : 


1- Show that a= Les y, with p, yr 


2- Deduce from this formula of a its variance V(a) ". 


Method 2 : 
Use the propagation of standard deviation formula. 


For simple linear regression, is it possible to find a, b, 
Aa and Ab using the generalized regression matrix 
method? 


Exercise 19 : Expectation of a 
Answers (Complete Edition) 


For linear regression, we denote a and B as the 

parameters of the population: E(y,)=a x;+ B. 

a and b are the estimated parameters from a samp! 
jzax+b 

Show that we have an unbiased estimator for a, so 

E(a)=a. 


16 mary: ECX+Y) 
variables: VIX+Y 


(CX) + BY). If X and Y are two independent 
(X) + V0). 
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Exercise 20 : Standard deviations 
proportional to y 
‘Answers (Complete Edition) 
We consider the particular theoretical case where the 
standard deviations of the linear regression are 
proportional to y : 0, =Ky,. This case, where the 
relative uncertainties are constant, is experimentally 
common, We are interested in the slope a. 


xyayt 


1- Show that: 


2- Express 


3- Calculate s, using the expressions found for the 
following two datasets: 


4 1 2 3 4 5 6 7 
wo | 1s [| 20 | 25 | 30 [ 35 | 40 
8.286 [17.286 | 18.286 | 27.286 | 28.286 | 37.286 | 38.286 

(We let k=0.1). 


Find these results again with the numerical method 
(evaluation of the derivatives with the small variations 
method). 


Compare the values obtained by the classical method. 


Exercise 21 : Interpretation of 
the expression of w, 
Answers (Complete Edition) 


Graphically justify the position of a in the expression 
of wi. 
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Non-linear regression 


Exercise 22 : Decomposition into Gaussians 
Answers (Complete Edition) 


A factory manufactures nails. With a caliper, we 
measure the size of 48 nails: 


Size 59.97 59.98 59.99 60.00 60.01 60.02 60.03 60.04 60.05 
qatiy] 2 [ 4 [6] 5] 5 [w] 9 ]ola 


Two machines manufacture the nails. The nails made 
by one of the machines are not exactly the same size. 
We assume that the size is distributed according to a 
Gaussian distribution. Determine the parameters of 
the Gaussian distributions of the nails manufactured 
by each machine (maximum, mean and standard 
deviation). How many nails have each machine 
produced? 


To check your correlation calculations you can use the 
spreadsheet function of 000 : 
COEFFICIENT.CORRELATION(**). 

Courbes : Insertion>Diagramme...>X¥(Dispersion)>etc. 
For matrix calculations, inversion of _ matrices 
INVERSEMAT(*), product of matrices : PRODUITMAT(*;*"). 


You can use the file IncertitudesLibresO0032.0ds on the 


website www.incertitudes.fr to realize regressions. Sheet 2, 
simple regression and sheet 3 with error bars. 
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III. PROBABILITY DISTRIBUTIONS 


We list different laws of discrete and continuous prob- 
abilities. We verify they are distributions of probability and 
we give their expectation =E(X) and variance 
V=E[(X-u)’]. 

‘The variance can also be calculated with the formula 
V=E(X*]—E[X}° and then we have the standard devia~ 
tion o= 


We can calculate other moments that allow us to de- 
termine, for example, the symmetry and the flatness of the 
probability distribution. 


Moments: 1,=£(X*) 


Normalized moments are instructive because they charac- 
terize the form of the distribution: 


‘Skewness (third standardized moment) 


B, : Kurtosis 


We also consider the sum of independent random 
variables: Z=X+Y . 
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A. Discrete Random Variables 


1) Binomial distribution 


We consider the number of successes in a sequence of 
n identical and independent experiments. Each trial has two 
outcomes named success and failure ( Bernoulli trial). Each 
success occurs with the probability p=P(S). The two 
parameters aren and p and we write ‘B(n,p). We want to 
determine the probability to obtain exactly k successes out 
of n trials. A path containing k successes has n-k failures 
and its probability is p'q"", where —p is the 
probability of a failure. 

Then we have to count the number of paths where k 
successes occur, there are different ways of distributing k 
successes in a sequence of n trials. n choice for the position 
of the first success, n-/ for the second and n+J-k for the 


—k)=n!/(n-k)! possibilities”. 
After we divide by k! to remove 
multiple counts (for example 
S,S.F and $,S,F correspond to the 
same path). From where: 


P(X=k) = (?) pq’ * 


(n=k)Ik! 


(often read aloud as « n choose k ») 


and ¥ P(x=K)=1 


nirials & 


17 nf, is said « n factorial » with n!=1X2x3...xn 
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we show that E(X)=np and V(X)=npq. 


‘The sum of two independent binomial laws of the same p is 
also a binomial law. The sum of a B,(n,,p)and a By{ity,p) is 
a B,(ns+p) 


To determine the distribution of Z sum of X and Y 
we use the following property for discrete distributions: 


P(Z=k)=> P([X=i]n[Y=k-i]) 


2) Geometric distribution 


We consider a random experiment with exactly two 
possible outcomes : S="success” and failure"; 
P(S) and q=1—p=P(F). We repeat independent 
trials until the first success. Let X be a random variable of 
the number of trials needed to get one success. We denote 


the distribution by G(p) 


P(x=k)=q'p , YP(x=K=1 


E(X)=1/p and V(X)=q/p° 


+ Example: We have a box with two white balls and a black 
ball inside. The balls are indistinguishable to the touch. We 
draw a ball, if we get a black ball we stop, otherwise we 
return the ball to the box and we repeat. What is the 
probability of drawing the first black ball in the third draw? 
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Answer: P(X=3)=(2/3)1/3=4/27 . 


+ Application: This distribution can be used like a discrete 
model of lifetime. For example, consider a cell which every 
day has a chance in ten to die. If it did not die the next day, 
its probability of dying the next day did not change: there is 
no aging". Every day the cell can escape death and it can 
live forever. Rather than making a decision every day, we 
can decide his death every hour with a probability q' of 
survival, like this q'*=q and q'~0.996. Or even a 
choice every second, we then have different geometric 
distributions witch modelize the same reality with p30 
and q>1- 


WO — Exponential distr. f(t) = exp(- 20) with Ae1.15741x10% 
10xi0* \ 
\ + Geometric distr: p,=(1-p'p 
} Dots every 86400 s 


for p=1.15741x10* and t=1s 


Ox 
id 10d 
Yo 


From a discrete distribution we reach a continuous one. 
The elapsed time since the beginning of the experiment is 
t=kAt, 

Let’s look at the cumulative distribution function: 


18 The geometric distribution has a remarkable property, which is 
known as the memoryless property. 
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P(X<K)=D ft pad etl p= pe 


We have used a Taylor series, indeed t>>At and k>1. 


Be 
Then P(X<k)>f f(t)dt, YL pe ™ af re*adt so, 
if we look at the limit, the geometric distribution becomes 
the exponential one with 4=p/At . Here for our cell % is 
about 1.16x10° per second. 


The sum of two independent geometric random variables 
Gip) is a negative binomial distribution BA(2,p), rank for 
obtaining two successes. 


For a binomial distribution the number of trials is fixed and 
we look at the number of successes, in the case of the neg- 
ative binomial distribution it is the opposite we are inter- 
ested in the number of trials necessary to achieve a number 
of successes fixed in advance. BNr,p) is the probability 
distribution of the rank of the r-th success", 


Then G(p)=BNULp! 


3) Poisson distribution 


The Poisson distribution expresses the probability of a 
given number of events occurring in a fixed interval. The 


19. There is also another definition: number of failures preceding the r- 
th success. The relations remain true by redefining the geometric 
distribution in the same way. 
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events occur with a known frequency independent of the 
time elapsed since the previous events”. If 4 is the number 
of times the event occurs on average over a given interval 
then the probability that the event occurs k times over this 
time interval is: 


p(xsk)=Be? , ¥p(x=k)=1 


E(X)=% and V(X)=2 


+ Example: During the Perseids meteor shower of 
‘August 1911 the hourly rate of shooting stars was 
50. What was the probability of seeing exactly 7 
meteors in 12 minutes? 


Answer: 


=P so- =7)=10.--0. 
R= GpSO=10 and P(X=7)=T pe "=9% 


The sum of two independent Poisson distributions Px (21) 
and Pr(A2) is a Poisson distribution Pz(A1+A2). 


20 or space traveled 
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B. Continuous Random Variables 


1) Uniform distribution 


To determine the distribution of Z sum of two 
continuous independent random variables X and Y we 
use a convolution: 


fbd=f fy)fyo—yey 


Consider the sum of two independent uniform distributions 
‘U(a/b). The integrand is nonzero if: 
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a<y<b and a<x-y<b then x—b<y<x—a 


If 2a<x<a+b: 


If atb<x<2b : 
If 2a<x<atb 
iy fp 2 x-2a 
then =f— ae 
tab) SEP O= (bal 
x<2a: f,(x)=0 


2a<x<a+tb:f,(x): 


a+b<x<2b:f,(x) 


x>2b: f,(x)=0 


We obtain a triangular distribution: 


2) Exponential distribution 


if 


‘The exponential distribution satisfies the memoryless 
property: P(T>b)=P,,,(T>a+b) . This distribution is 
widely used to model waiting times. It is also used to model 
lifetime without aging : like the lifetime for a particle 
decay. 


‘The mean life E(T) and the half-life tia, P(T>t,.)=0.5 
. are two different times. 


The distribution of the sum of two independent exponential 
distributions is not an exponential distribution. 
3) Normal distribution 


The normal or Gaussian distribution has previously been 
described page 20. 


The sum of two independent Gaussian distributions 
Ny(uo7) and Ny(u2,027) is the Ga 
Nelurtp2, or +62"). 


4) Student's t-distribution 


The t-distribution is express with the number of degrees of 
freedom k and the gamma function (function described in 
the mathematical tools). 


‘As the number of degrees of freedom grows, the t-distribu- 
tion approaches the normal distribution (0,1): 
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. k-2 
Kurtosis: B=35—G if k25. 


In exercises the expressions of the first Students are ex- 


pressed, we show that fart f,(x)dx=1, we compute the ex- 


pression of the variance and finally we show on examples 
that the sum of two independent Students is not a Student. 


1 


First Students 


Kae] ov Be] Be 7,10) 
rf2]- : 7 =0.318 
313-|~=, 7 : 354 
3 3 : : 366 
r; 5 = 7 =0.375 
slo sa 9 7 °| 20.380 
=1.67 
o]7] 32 6 : 0.383 
=15 
Tl] 75 5 | 125 =0.385 
=14 
spo] 43 | 92 [ors 0.387 
2133 | =45 
afi) o7 [as [49 | ae | ey 
129 | =42 Tosa /\!*5] 
fn] sa 4 
£1.25 
nf] ws | 27 
=122 | =39 
= 1 3 5 | waa e*? | =0399 
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5) Chi-squared distribution 


Let consider k independent normal distributions 
NiO,1):T), Ts,... and T;. The sum X,=T7+T?+...4T2 
follows a 72-distribution with k degrees of freedom. 


For k=1 and x20, filX)= oo 


“FPP IKI2) 


Expectation: E,=k Variance: V,=2k 


VBTk 


Skewness : Biu= 


Kurtosis: B,,,.=3+12/k 
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The chi-squared distribution converges to a normal distri- 
bution for large k . Normal distribution of expectation k 
and variance 2k . 
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C. Function of a continuous distribution 


We have a random variable Y defined as a function of a 
continuous random variable X: Y=@(X). 

We know the law of X and we want to determine the law 
of Y. We use the cumulative distribution functions F and 
then we consider the derivative of Fx to get the probability 
density function f. 


Cumulative distribution function of X : 
F,(x)=P(X x)= J fx(x)dx 
We consider the case where @(x) is a strictly monotonic 
function. Then Y is a continuous distribution. 
Cumulative distribution function of Y : 
dFy(y) 
dy 


Fy(y)=P(Y¥<y) and fyly 


We try to express Fy(y) with Fx. 


1) Case where (x)=In(x): x>0 and Y=In X 
Fy(y)=P(Y¥ <y)=P(InX<y)=P(e’ 


"*<e")=P(X<e’) 


We have applied the inverse function @"(x)=e". 


The exponential function is strictly increasing, so the direc- 
tion of the inequality has been preserved. 


so Fy(y)=Fy(e’) and fy(y)=e”fx(e’) 
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+ Example: X is a uniform distribution (1:2), What prob- 
ability distribution is Y? 


0 x<1 
We have fx(x)=| 1 1<x<2 then if e’<1, y<o and 
0 x22 


fy(y}=0. We continue like this for the two other cases 
and we obtain the distribution of Y: 
y<0 
O<y<in2 
y2In2 


2) Case where y(x)=ax+b: a#0 and Y=aX+b 
If a>0: 


yob 


Fy(y)=P(Y <y)=P(aX+b< y)=P(X= a 


A linear function is strictly increasing for a strictly positive, 
the direction of the inequality is not changed. 
<b 1, ;y-b 


Then Fy(y)=Fx(~ 


Ifac0: 
Fy(y)=P(x22— 
and fy(yJ=— 
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« Example 1: X is a uniform distribution 7(0,1). What 
probability distribution is Y? 


oO x<0 
We have fx(X)=| 1 O<x<1, so if 
O x21 


a>0 then y<b and fy(y)=0. We continue like this for 
the two other cases and we obtain the distribution of Y: 


0 y<b U(0,1)>U(b,a+b) 


fry)= b<y<a+b 


| a If @(x)=(b-a)xta and acb: 


0 y>a+b U(0,1)>U(a,b) 


+ Example 2: X is a Gaussian distribution (0,1). We can 
find again a A(t) distribution with Y= OX+. In gen- 


eral, by applying an linear function we obtain a distribution 
of the same kind. 


3) Case where g(x)=x’: Y=X* and y>0 


=F,(Vy)-Fy(-vy) 


Fy(y)=P(X°sy)=P( 


(fe(Vy)+fx(-Vy)) if y>0 else zero. 
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4) Case where g(x)=e':,  Y=e* and y>0 


F,(y)=P(e*<y)=P(X<Iny)=F,(In y) 


and fr(y)= 


x(Iny) if y>0 else zero. 


y 


D. Numerical simulation 


We simulate continuous and discrete probability di 
tributions using computers. For this purpose we use uni- 
form distributions created by random number generation 
algorithms: 


Continuous uniform distributions U(0,1) : Ran# on a 
pocket calculator, ALEA() in the LibreOffice spreadsheet, 
ete. 


Discrete uniform distributions UG,j) : for example, 
rand(,j) in PHP language. rand(0,999)/1000 simulates a 
uniform continuous law discretized to the thousandth. 


+ Inverse transform sampling : the inverse transformation 
method takes uniform samples between 0 and I from U. 
We express a X distribution as a function of U by inversion 
of the cumulative distribution function Fy. With F strictly 
increasing: X=F '(U). 
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Case of an exponential distribution: fy(x)=%e* if x>0 
else zero, then F,(x)=f fy(x)dx=1-e “=y if x30 


clse zero, So x=—In t=) 


and finally we simplify, 


x 
knowing that /-U and U have the same distribution. 
' ss Inu 
Simulation of an exponential distribution: x-=— 


* Simulation of two independent normal N(0;1) distribu- 
tions X, and X? from two independent uniform U(0,1) dis- 
tributions U; and 


x, 


=2InU,cos(2xU,) 


X,=-2InU,sin(2xU,) 


‘This is the Box-Muller transform. For a Gaussian there is 
no direct formulation of the cumulative distribution fune- 
tion F. Thus F" has no simple analytic expression. 


* Method for any X distribution: 


If we have a continuous distribution we obtain a discrete 
distribution using class intervals for x. We then have a his- 
togram and each bar of height p: is also discretized in units. 
We get an outcome of the distribution X drawing randomly 
and equiprobably a unit of the histogram. We get by 
putting the bars end to end a full range of units. We use a 
discrete uniform distribution UCN) where N is the total 
number of units in the histogram. The generated value is 
compared to its position on the range and we get a x: obser- 
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vation. 


Discrete Continuous 
distribution 


P 
‘YB aistribution 


4 Range 


pee __see |e 


% 


* There are many other methods that use the different prop- 
erties of the probability distributions. For example, by sim- 
ulating the Bernoulli distribution, we obtain, by sum, a bi- 
nomial distribution which itself allows us to simulate a nor- 
mal distribution. 
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E. Exercises 


Exercise 1: Binomial distribution. 

Answers (Complete Edition) 
Check that the binomial distribution defines a 
probability distribution, Calculate the expectation and 
the variance as a function of the parameters n and p. 


Exercise 2: Sum of binomial distributions 

Answers (Complete Edition) 
Show that the sum of independent binomial 
distributions with the same parameters p is itself a 


binomial distribution whose parameters will be 
determined. 


Exercise 3: Geometric distribution 

Answers (Complete Edition) 
Check that the geometric distribution defines a 
probability distribution. 
Calculate the expectation and the variance as a 
function of the parameter p. 
Show that the sum of two independent geometric 
distributions with the same parameter p is a negative 
binomial distribution NB(2,p). 


Exercise 4: First successes 

Answers (Complete Edition) 
1- Let consider a balanced coin. What is the 
probability that the first tail will appear on the fifth 
toss? Knowing that the first tail has not yet appeared 
on the third throw, what it is the probability that it will 
appear for the first time at the eighth toss? 
2- Let consider a balanced die. On average after how 
many tosses appears the first six? What is the 
probability that the first six will appear in the first six 
throws? 
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Exercise 5: Poisson distribution 

Answers (Complete Edition) 
Check that the Poisson distribution defines a 
probability distribution. 
Calculate the expectation and the variance as a 
function of the parameter A. 
Determine the sum of two Poisson distributions. 


Exercise 6: Uniform distribution 
Answers (Complete Edition) 


Calculate the variance for a continuous uniform 
distribution T(a,b). 


Exercise 7: Exponential distribution 
Answers (Complete Edition) 


Check that the exponential distribution defines a 
probability distribution. 

Calculate the expectation and the variance as a 
function of the parameter A. 

Determine the sum of two exponential distributions. 


Exercise 8: Sum of Gaussians 
Answers (Complete Edition) 


1- Determine the sum of two standard normal 


distributions A(0,1). 
2- Determine the sum of two normal distributions. 


Exercise 9: First Students 

‘Answers (Complete Edition) 
1. Give the expressions of the first Student functions. 
2- Give the polynomial degree in denominator, the 
variance, the kurtosis and the y-intercept. 
3- Give the expression of the Student for k=9, 
centered and with a variance equals to 14/5 
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Exercise 1 


Student's t-distribution 
Answers (Complete Edition) 


Whatever k, show that the Student distribution 
corresponds to a probability distribution. 


We can use the integral ,= f 


+1 
— 


dx and 


carry out an integration by substitution with uw 


Exercise 11: Variance of Student's t-distribution 
Answers (Complete Edition) 


Determine the variance of a t-distribution. 


Exercise 12: Sum of Student's t-distributions 
Answers (Complete Edition) 


Using an example show that the sum of two 
independent t-distributions is not a t-distribution. 


Exercise 1 


: Chi-squared distribution 
Answers (Complete Edition) 


Give the expressions of the first chi-square density 
functions in the cases where k: and k: 


Exercise 14: Product of distributions 
Answers (Complete Edition) 


Let X and Y be two independent distributions. 


1. Propose a general method for determining the 
probability distribution of Z=XY. 
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2. We now consider the case where X and Y are two 


independent continuous uniform distributions T(1,2). 
a. Determine the analytic expression of the Z 
distribution. 

b. Find the shape of probability density function of Z 
with a numerical simulation of the product on a 
spreadsheet for n=10,000. 


Exercise 15: Sum of exponentials 
Answers (Complete Edition) 


Let X, be n independent exponential distributions with 
the same parameters A. Let S,, be the distribution of 
the sum: S,=X,+X,+...+X,. We have too: M,=S,/n. 


1, Determine the probability distribution S:. 
2, Determine the probability distribution S), 
3, Determine the probability distribution M,. 


Exercise 16: Inverse distribution 
Answers (Complete Edition) 


Let X be a random variable with strictly positive 
support. 
1. Determine the probability density function of Y=1/X. 
2. a. Find T, the inverse distribution of M, defined in 
the previous exercise. 
b. Find the inverse distribution of the Cauchy 
distribution: 


Cauchy distribution X: f,=+ 


for all x 
14x’ 


124 


125 


ESTIMATORS 


We have a random variable X that follows a known 
distribution with parameters that are unknown, 
We have a n sample (X1,X2,....X») of the random variable 
X from a population. We want to have a method to estimate 
at best the different parameters that define our distribution 
(inferential statistics). Our parameters are denoted by the 
letter 0 and we call 7, our estimator of 0. 


At first we use the sample to give a point estimate of 0, 
that has a high probability of being close to the parameter, 
next we provide an interval estimate of @ that has a high 
probability of containing 0. 


A. Properties of an Estimator 


1) Bias 
The bias of an estimator 7; is the expectation E(T,—0) . 


Then: by (0)=E(T,)—0 


The estimator is unbiased if b,=0 so E(T,)=0. 
It is best to have an unbiased estimator if not 


asymptotically unbiased: lim b,=0, 
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2) Mean Square Error 


We can compare different estimators with their mean 
square errors. The mean square error of 7; is defined as the 
expectation E[(T,—0)] and we show that: 


r,(O)=V(T,)+b" 


Indeed E[(T,—0)]=E(T,)-20E(T,)+0° (linearity of 
the expectation) and eventually after simplifications of 
r=V(T,)+E(T,)'—20(b+0)+0° we find the previous 
expression. For an unbiased estimator its mean square error 
is equal to its variance. It is useful to choose an estimator 
whose mean square error tends to zero with the sample size 
and the faster is the convergence the better is the estimator. 
X,, is the sample average regarding the 
») defined as: 


1 
==>) x, 

Demonstrate that X, is an estimator of the expectation 
0=m=E(X) . Study its properties: bias, convergence. 


+ Example 
sample (X,, 


T,= 


E(T,)=1nd E(X,) using linearity of the expectation 
then E(T,)=1/n.nm=m and b=0. 


V(T,)=U/n DV (X,) (variance of a sum with 
independent variables) then r=1/n’.no* and r=o"/n. 


The sample average is a good estimator of the expectation 
of a random variable. The estimator is unbiased and the 
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‘mean square error tends to zero when n tends to infinity. 


+ Example 2: We are looking for an estimator of the 
variance of a random variable, we propose the following 
‘two estimators: 


Which, in your opinion, is the best estimator of 0” ? 


SP=Und (X?-2X,X,+X,)) and 
S,7=1n YX 7-27, X/n+T2=1Un yd, X7-TR 
(X?)-E(T;)=07+m?-V(T,)-E(T,)° 


o and b, 


By a quite similar calculation we find b 


This quantities are two estimators of the variance because 
they are unbiased or asymptotically unbiased. 

2 ; : 
Ry. is the best estimator because it has no bias. 
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B. Construction of estimators 


We want to have a general method to find the 
appropriate estimators to estimate the parameters of a 
distribution, 


1) Method of Moments 


We identify the moments of the population with those 
of the sample. We consider as many moments as we have 
parameters starting with the first moment. As we will see 
on examples this method provides us with estimators but 
does not guarantee us that these are the best in terms of 
bias and mean square error. 


Theoretical moments: m,=E(X"), kiN" 


Sample moments: xy=t Sixt 


We can also consider the moments centered or com- 
pletely normalized, the approach remains the same. The 
first n-sample moment is the sample average and it corre- 
sponds to an excellent estimator as shown before. On the 
other hand, the centered second moment has a bias, as we 
have shown before we should divide by n-/ instead of n to 
have a unbiased estimator. 
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+ Example 1: We have a checkerboard of 100 squares and 
200 seeds. Every second 
we randomly place a seed 
in one of the squares. At 
the end of the experiment 
we count the number of 
seeds in each square and 
we count the number of 
squares that contain no 
seed, one seed, two seeds 
and so on, Let X be the 
random variable for the 
numbers of seeds per 

square. We obtain the following distribution: 


k}/o]}/1f{2}3],4]/s];e6},7]8 
n | 12] 28]34]10] 8] 7} 0],1]0 


We assume that this distribution follows a Poisson 
distribution. Deduce the value of the parameter i. 

The parameter of this distribution is equal to the 
expectation therefore according to the theorem of 
moments, 2 is estimated by the sample average: 


This result is consistent with our model, indeed let us name 
nthe number of squares and N the number of seeds. We 
have a uniform random distribution. We could, for 
example, use two balanced ten faces dice of different colors 
for the horizontal and vertical position. The frequency of 


21 Using the second moment we will find 4=2.06 
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the event per second for each square is I/n. For a Poisson 
distribution at any instant the event can occur, here the 
time is discretized, nevertheless the approximation of a 
continuous time is correct because one second is a small 
duration compared to that of 200 seconds of the 
experiment. So we can take A=N/n. We tend to a 
Poisson distribution when the number of squares and seeds 
tend to infinity. 


+ Example 2: We assume that the lifespan of a glass follows 
an exponential distribution. We observe in months the 
following lifetimes: 7, 5, 37, 35, 17, 9, 6, 13, 4 and 8 
months. Determine thee parameter i of the distribution. 


For an exponential distribution E(T)= 


the first moment: ) 


We thus have a point estimate of the parameter 4 and the 
life expectancy is about 14 months. 


We will show in an exercise that this estimator T, has a 


bias: E(T,)=—2yh and by: aa F 
This bias and its mean square error tend to zero: 
oe (n+2) 
(n=1)(n=2) ° 


We construct from this estimator a new estimator W,, 


without bias: W,="—t7,= 254 
ee 


We show that E(W,,)=A and ry =V(W,)=2'/(n—2) . 
This estimator is better than the previous: zero bias and 
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lower risk (mean square error). A new estimate of the 
parameter gives 2~0.064 and the life expectancy is about 
16 months. 


+ Example 3: We consider that the mass of the apples 
produced by a tree follow a normal distribution. We 
randomly draw and measure the following masses in grams: 


158 |131 [146 [158 [125 [153° |166 | 121 
127 |123 [195 149 }124 [153.123 -}129 


Determine the mean and standard deviation of the 
distribution. 


a2 ma=125, ot 


* Example 4: The planks produced by a sawmill have a 
length which follows a uniform distribution Z(a,6) 

We measure the lengths in millimeters of 8 planks drawn at 
random: 2017, 1987, 2018, 2014, 2003, 1985, 2013 and 
1981. Estimate a and b. 


)? _a?bab+b* 
3 


then a+b=2X," and ab=4X,' 7-3X7 


Eventually a~1977 and b~2028 
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2) Method of Maximum Likelihood 


Let X be a discrete or continuous random variable. We 
want to estimate a unknown parameter @ from a set of 
observations {xi}. 


We define a function f(x,6) such that: 


| P,(X=x) for a discrete variable 
flx,0) = | s 


| folx) for a continuous variable 


We define the likelihood funetion L. Thi 
6, determine by the numbers x),.2 6.5; 


a function of 


L(x, Xy yeep X,pe0eoXy 8) = F(X, 0)X f(y 0)X..-X F(X, .0) 


also we can simply write: 


Lx, X25 


Kipp Xqe 8) = Tlf (x,0) 


The value of the parameter for which the product of the 
probabilities, or probability densities, taken at the 
different points of the sample, is maximum, is considered 
the most probable value. The maximum likelihood 
estimate of 0 is the value that maximizes the likelihood 
function L(0). 

The maximum likelihood principle is simple and the 
method is easy to implement: 


OL(x,,0) Pt &L(x,,0) 6 


30 i" 00° 
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Usually on can find the maximum by differentiating the 
likelihood function L(0). However the calculation of the 
derivative may be tedious, it is why we prefer to consider 
the logarithm of £(0). The logarithm is a increasing 
function and we can consider the extreme values of 
In(L(Q)) instead of L(0). 


+ Example 1: Let us take again the case of a Poisson 
distribution and determine the estimator of 2 by the 
method of maximum likelihood. 


We find the same estimator as by the previous method. 


+ Example 2: Let us take again the case of an exponential 
distribution and determine the estimator of % by the 
method of maximum likelihood. 


f.lt}=ae™ and L=[[ae™=a"e © 
iM 
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then 


‘The same estimator as by the previous method. 


+ Example 3: Let us take again the case of a Gaussian 
distribution and determine the estimators of 1 and o by the 
method of maximum likelihood. 


so Inb=-nino~Zin(22)-25 (54 } 


We obtain the same estimators as by the moment theorem 
and we have again an estimator of the variance biased. 
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+ Example 4: Let consider now the uniform distribution. 


= if {xJc[a,b] 
(x,a,b)= Tis. a,b)=| (b-a) 
0 else 
For a fixed, smaller is b bigger is the likelihood, then 
b=max({x, 


For b fixed, bigger is a bigger is the likelihood, then 
a=min({x,|) . 


We have here very different estimators than by the method 
of moments, for the example of the planks we have the 
following estimates: @=1981 and b=2018 . 


These estimators are biased, for example b is necessarily 
smaller than the theoretical value, but, contrary to the 
method of moments we are assured that all the x; belong to 
fa, b}. 
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C. Interval estimate 


We now have effective tools to determine the values of 
the parameters of a distribution. We have determined point 
estimates and now we want to determine a confidence inter- 
val. 


‘The central limit theorem allows, from a large sample, 
to estimate the mean of a probability distribution with a 
confidence interval. 


But what about the other parameters different from 
the mean? For example, what are the uncertainties on the 
parameter } of an exponential distribution, or the bounds 
and b for a uniform distribution’ 


We consider an unbiased estimator and if the estima- 
tor is biased we create a new estimator by removing the 
bias. 


We use three different methods. The integral method which 
requires to determine the full probability distribution of the 
estimator. A second method, simpler, provides an inequal- 
ity that overestimates our uncertainty, but only requires 
knowledge of the variance of the estimator. And finally a 
third by numerical simulation. 


in- 


For the second method we use the Chebyshey 
equality. This inequality can be applied to any probability 
distribution in which the mean and variance are defined. 
We do not have to know the aspects of the estimator distri- 
bution, only its variance, but the inequality generally gives a 
poor bound: 
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Ver0, P(|X-E(X)|ze) < v(x) 
€ 


+ Example 1: Consider again the checkerboard and the 

Poisson distribution with parameter 2. This parameter is 

estimated by the mean and we can therefore, in the case of 

large numbers, use the central limit theorem: 
hEhgtt,oln 


Let us estimate the variance: 


k fofif2[3]4]s[o]7 
kaw |-2]/-1fo0]1]2[3[4]5 | 6 
(kin | 4] 1 [0 | 1] 4 | 9 | 16 | 25 | 36 
an [12] 28]34[10] 8|7[0 [1 ]o 


» 0.21.44 and 4=2.0+0.3 with 


95% confidence. 
Here the sample size is not sufficient for this method but 
we have illustrated the general method for large samples. 
We probably have underestimated the width of the interval. 


* Second example: The waiting time of a train follows a 
uniform distribution U(0,a). We observe the following 
waiting times 3, 12, 7, 17, 8, 14, 2, 5, 10, 14, 15 and 11 
minutes. What is the value of a ? 

What is the bias, the variance of the estimator T, and the 
uncertainty on a at 90% confidence? 
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a) Let us first use the method of maximum likelihood 
which gives us the estimator 7, , the distribution of the 
maximum of a number 1 of independent, identically 
distributed variables: T,=max((X,}) 

ais here estimated at 17 minutes. 


The distribution of the maximum is such that: 
P(T,Sx)=P([X,Sx][X,=x]N...9[X,<x]) 
the variables are independents, then: 


P(T,<x)=] | P(X,<x) 
cy 


Cumulative distribution function of X: Fy(x)=P(X <x) 
for 0<x<a 


F,(x)=0 if x<0, Fy(x)=1 if x>a 


dFr(x)__ x! 


So we obtain the density of T. : fr,(x)=—Gr 7 


for 0<x<a and f;,(x)=0 else. 


Expectation: £(T,)=f xf;,(x)dx= 


The estimator 7, is biased: b; =—"—a—a= 
sont+l n+1 


a is underestimated and the bias is asymptotically zero. 
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a 
42° (nell (n¥2)(n¥iP 


It is better to take an unbiased estimator, for this we re- 
move the bias and we get W,: 


= Mtl with by =0 and V(W,)= 
n : nne2) 


New estimate of a with W,: a=18.4 minutes. 


For the uncertainty Aa we will compare three methods. 


-> With determine a confidence interval with the Cheby- 
shev's inequality: 


We apply the inequality to W,.: P(|W,,— 


so P(|W,, 


and P(e>a—W,>-¢)=P(U,+e=a=W,, 


we set € and we have: 
1-P(W,-e<a<W,+e)<a 


Eventually: 
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In the 90% confidence case «&=0.1 and for our sample 


Then 13.9<a<22.9 and a=18.4+4.5 minutes. 


-> Let us determine a confidence interval with an integral 
calculation on the probability density of the estimator. 


We determine the probability density of W,: 


n ny n 
pr MoS pay XP (Ins gy *) 


and fy (x)=0 else. 


We have an asymmetric probability distribution whose 
maximum corresponds to the upper bound. For a 90% con- 
fidence we remove the 10% of the left distribution tail from 
the confidence interval. 


Thus we define dar aNd diniy SUCH aS? 


+1 
Upper bound: ngs = a 
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as 3] X17=19.95 and a,,,,~20.0 minutes. 


Lower bound: J fu (x)dx=1-0 
By numerical calculation with a= 18.4 and n=12, 


2 
J fw,(x)dx=0.9 with a,,,~16.4 minutes. 


In conclusion, a~18.4*) minutes 


with 90% confidence. 


The interval found is asymmetrical. The bounds are in- 
cluded into those of the Chebyshev's inequality and we 
have here a more precise estimate of a. 


-> Let us now perform a numerical simulation on a 
spreadsheet. The ALEA() function of the spreadsheet pro- 
vides real randomly and uniformly distributed between 0 
and I. Then we multiply by the point estimate of a to ob- 
tain the distributions X=U,(0,a). We generate 10,000 sam- 
ples of size 12. We place the maximum of each of these 
samples on a graph and thus have the sampling distribution 
of Ty: 
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Estimator unbiased bana tan the Maximum Lkethood 


‘Simulation of Wn with N=10.000 and 12, 


OL 24S ETEVMUNUUMH ET BBMARBAB 


We thus find the same results as by the preceding method 
(file : www incertitudes.fr/livre/Train.ods). 


a) We now use the estimator of the moment theorem: 
T,=2X, . Here, for large n, we can use the central limit 
theorem because an affine function of a distribution gives a 
new distribution of the same form. We estimate the mean 
with the Gaussian law: 


—_ 3412+..411 


= id 
i 9.83 an 


9.8+2.3 with 90% confidence 
Then a=19.7+4.6 minutes with 90% confidence. 


m= 


tt, s/v 


In this case the value of 1 is only 12 and the distribution of 
the population is not normal, we want nevertheless to show 
how one would proceed for m large. As we will see with 
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numerical simulation, the interval is thus underestimated. 


Indeed by carrying out a numerical simulation with this 
estimator, a=19.7+5.5. minutes with 90% confidence: 


os 
nae dsaret th te erect Mone 


Shy Th we 2 = 
on f 
at 
ow 
os 
oon 
ew 
ms 
Pov? DUUGENE HTB BRaaaH EBT ADD 
Comparison of the results of the different methods with 
90% confidence: 

Chebyshev’s inequality —}————»_____> 
dais Fcc eT aL) 
Takelihond Integrals and 

Simulation 164 18.4 20.0 
Central Limit Theorem 
=————— 
Method (Sample too smal) 181 197 4a 
of 
Moments Simulation = —- py, 
a 17 2 
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In conclusion, the maximum likelihood estimator converges 
much faster than that of the moment theorem and we 
prefer this first method. The variance converges to 1/n? 
instead of I/n: 


‘ fespce 
vi(w,) > and [V(T,) sye=>— 
— = 
70 
. Convergence of variances 
60 
\(n) Method of Moments 
50 myn) 
. (n) Maximum Likelihood 
Paes oun) 
30 . 
20 fos, 
. "a 
10 . toe 
tee Se eeeeeeeeae 
. oo 
‘ SFr eeressressesconcs 
° 5 15 HD 
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D. Exercises 


Exercise 1 : Estimators of the mean 
Answers (Complete Edition) 


Let consider the (X;, X2, X;). X;, X, and X; are three 
independent variables with the same distribution, 
expectation m and variance o*. 

Compare the following three proposed estimators to 
estimate the mean m of the sample: 

Ag=(X$X,+X,)/3, By=(Xi+2X,+3X,)/6 and 

C= (Xi +2X2+X3)/3. 


Exercise 2 : Homokinetic Beam 
Answers (Complete Edition) 


We consider a homokinetic beam of C* ionized 
carbon atoms. We measure the momentum and the 
Kinetic energy of 100 atoms of the beam. The beam is 
considered perfectly unidirectional. 

The total momentum magnitude and Kinetic energy 
are: 


p=), mv,=2.418X10 ” kg.m/s and 


=1.518X10 "J with m= 


993x107" kg. 


Let V be the probability distribution of the speed of 
the ions. Determine the average speed v,,, its variance 
oy? and the uncertainty Av with 95% confidence, using 
the sample taken and the appropriate estimators. 
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Exercise 3 : Two estimators 


Answers (Complete Edition) 


Let X be the following discrete random variable: 


Values 0 1 2 ] 
Probabilities 30 8 1-46 | 


1. What values of @ define a valid probability distribu- 
tion? 


2. Calculate E(X) and V(X). 
3. Let (X,, Xz,...,X,) be a sample of X. 


1< ~ 
aX and T, 


We have X,: X,+b. 


Determine a and b for which T. is an estimator without 
bias for 0. 


Determine V(T,). 


4. We now consider the random variable Y, defined for 
all i€[0;n] by Yi=1 if and Y.=0 otherwise. 


Let Z,=). Y,. Determine E(Z,). 
1 


Z, 
Show that U,=— is an unbiased estimator of @. De- 
n 


termine V(Us). 


5. We make estimates of @ with the following realiza- 
tions: 


Values 0 1 2 
Frequencies 31 12 57 


Estimate ®. Which estimator do you prefer? 
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Exercise 4: Ballot boxes Answers (Complete Edition) 


Two identical urns contain the same proportion p of 
black balls. In the first ballot box we draw with 
remplacement a sample of size n; and we note P; the 
proportion of black balls in this sample. We perform 
the same experiment for the second urn. 


P+P, 
We define T= 3 2 and U=x P,+(1—x)P, 


with x € ]0,1[ 


Show that T and U are two estimators of p. 
Which one is the best ? 


Determine the value of x to have the optimum 
estimator. 


Exercise 5 : Continuous variable 
Answers (Complete Edition) 


We consider the continuous 
variable X with the following if x1 
density: f(x= ae 


Where a is the parameter we : 
want to estimate (a>1). 0 otherwise 
1, Check that f defines a valid probability density. 

2. Calculate E(X). 


3. Determine estimators of a by the method of 
maximum likelihood and by the method of moments. 
4. Give point estimates of a for the following 
observations: 

1.16 / 1.80 / 1.04 /3.40 / 1.22 / 1.06 / 1.35 / 1.10. 
5. Can we calculate V(X)? 
6. Perform a numerical simulation of the law of X. 
What can we guess about the biases and convergences 
of the estimators found? 
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Exercise 6 : Linear density 
Answers (Complete Edition) 


Consider the following continuous random variable X: 


ie jax+b if O<x<1 
ae 


| 0 otherwise 


1. Express b as a function of a such that f defines a 
probability distribution. 


2. Calculate E(X) and V(X). 


3. Determine an estimator T;, of a by the method of 
moments. Discuss the properties of this estimator. 


4. We draw a sample: 
0.81 | 0.67 | 0.72 | 0.41 | 0.93 | 0.55 | 0.28 | 0.09 | 0.89 


Determine a point estimate of a. How would we get an 
interval estimate? 


Exercise 7 : Estimators for the exponential law 
Answers (Complete Edition) 


The exercise of the previous chapter on page 124 
provides the expression of the probability density of 
the estimator T, of A obtained in the course: 


he if x>0 and zero if not 


1. Determine the expectation, bias, variance and the 
mean square error of Ts. 


n-1 


2. Let W,=——T,, . Determine the expectation, bias, 
n 


variance and the mean square error of W, 
3. Which estimator would you recommend for A? 
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Exercise 8 : Decays Answers (Complete Edition) 


The law of probability X of particle decay as a 
function of time follows an exponential distribution 
with parameter 4. We measure the lifetimes in 
2.2 2.32 1.93 0.88 4.36 Microseconds of a sample 
71 06s 053 469 021 of ten particles and we 
ei Bo oy. want to deduce a point 
estimate and an interval estimate of 4. We will use 
different methods and comment on them. 


1. With the central limit theorem could you estimate 
the expectation m of X and its uncertainty with a 
confidence of 90% (m : mean lifetime)? With the 
propagation of uncertainties formula we then could 
find the value of A with its uncertainty. What do you 
think about this estimate of A? 


2. Let T, be the estimator of A found during the lesson. 

Like we shown previously this one is biased and then 
n-1 

weuse W,=——T,. 


Determine by an integral calculus the uncertainty on A 
with 90% confidence. 


3. Find now this result by numerical simulation. 
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V. Measure with a ruler 


Article published in the BUP [ii]. 


ABSTRACT 
The measurement of a physical quantity by an acquisition 
system induces because its resolution a discretization error. 
We are here concerned with measuring a length with a 
graduated ruler. This type of measure leads us to consider a 
uniform continuous probability distribution. We then use a 
convolution to determine the uncertainty with its confidence 
of a sum of lengths. Finally, we generalize to the general 
case of the calculation of uncertainties for independent 
random variables using the error propagation formula. 


INTRODUCTION 

We want to measure lengths and evaluate uncertainties 
as exactly as possible. Uncertainties about the measured 
values and their sums. We have a ruler of 15cm graduated 
to the millimeter and two sets of cards. The ruler is 
assumed to be perfect and the cards of each game identical. 
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1, MEASURE OF THE LENGTH OF ONE CARD 


We place the 
graduation zero on 
the left edge of the 
card. On the right 
edge we consider 
the graduation clo- 
a sest to the edge. 
c oo The experimenter 
| 7 does not read 
9 between the gra- 8 

duations. The thickness of the lines 

which delimit a graduation is considered negligible 
compared with the width of this graduation. We get thus 
for the deck 1: 


4 0.0S.cm. 
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Concerning the cards of the second deck: 
1.2£0.05cm. 


0 Xin Xm Xmax x 


We accept a loss of information due to the resolution 
6=1mm of the ruler. When we use these data later, all 


values between Xj, =X, 5/2 and Xpip=X_— 5/2 are 


equally probable. We have to consider a continuous and 
uniform random variable X. x is a realization of X. This 
distribution of probability has a range E=X,,.—Xpiy and 
its density f(x) verify: 


@ 
J f(xjax=t 

The probability to be between x and x-+dx is f(x)dx. 

The result is necessarily between Xj, and Xpyq : for 

example x, =8.4+0.05cm with 100% of confidence, but 

.4£0.04.cm with a probability of 80%. 
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To characterize the spreading of a distribution we consider 
the range E and the standard deviation o whose definition 
for a continuous law is: 


=f (x—x,,)’flx)de, 


V= 
V is called the variance. For a uniform distribution: 


o=d/V12=0.298, 


and we have *=%,+0 with 58% confidence. The 
standard deviation is an appropriate quantity to characterize 
the width of a distribution. The range is defined by the 
extreme values which may be unrepresentative or absurds. 


2, LENGTH OF TWO CARDS PUT END TO END 


We want to determine the uncertainty on x with 
X=X,+X, . If we plot ¥) asa function of %, the set of 
the possible points forms a square domain. The set of 
points such as x is constant is straight line segment of 
slope -1 and intercept x : X2=—X, +X . There is only one 
case where X=Xyiy then |Xr=%rmini X2=Xoin at point 
A in the figure. However on all the segment [CD] we get 
X=X,,. We understand that the different values of * do 
not have the same probability. 
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Xana & XQ e 

Xam 

Xan 
ry D 


min Kim Xmax x 
The probability density f of X is computed from those of 
X: and X2, For a sum of independent random variables the 
result is given by a convolution [ii] : 


X€X qu, f(X)=0 
(x— Xin) 


Kinin XSX y=Pf (2): 


Fx)=f hy) filx-y)ay = 


Kp XEX pac PF 0 


X>X gu? f(X}=O0 


We then have a triangular probability distribution. 
We obtain x=19.6+0.1cm with 100% confidence, and 
x=19.6+0.055cm with 80% confidence. 


3. ANALOGY WITH THE THROW OF TWO DICE 


For each die the six values are equally likely. Here the 
law of probability is no longer continuous but discrete. We 
launch two dice simultaneously, the sum of the values 
obtained is between two and 12. In this case, there is no 
equiprobability, a way to get two with a double one, two 
ways to get three with one and two or two and one ... to get 
seven we have the maximum of possibilities. We thus find a 
triangular distribution. 


0123456 X 0123456789 0NKR X 


4, LENGTH OF TWO CARDS OF THE SAME DECK 
PUT END TO END. 
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‘The cards of a deck are supposed identical then if the 
length of one of them is overestimated, it will be the same 
for the second one. In this case the errors are added and 
can not be compensated. For two different cards, the first 
measure can be underestimated and the second 
overestimated, and a compensation can occur. Here it is no 
longer the case and when X=X;+X;' we obtain a 
uniform distribution of width 26 . Our random variables 
are not independent, 


For the deck 1: 


8.4+ 0.04 cm=>x=2x,=16.8+0.08cm 
with a confidence of 80%. 


5. SUM OF N INDEPENDENT LENGTHS 


We have X X,. Each length X; follow a 


uniform distribution of width 4 . For the sum of nine 
independent random variables after iteration of the 
calculation we obtain the following curve: 


X 
92 ie X62 


In this case we obtain X=Xj0.1Lem with 80% 
confidence. With 100% confidence: X=X,,,)=0.45cm , 
which leads us to consider domains where the probability 
of presence of X is really negligible. An uncertainty of 
0.45em seems unnecessary while 99% of the cases were 
already present with an uncertainty of 0.22cm. 


Working with a confidence of 100% it is like considering 


the range, it is additive for a sum of variables. The range is 
proportional to N. 
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80% 95% 99% 
1| 0.406 | 0486 | 0.5085 
2/0555 [0.786 | 0.905 
3| 0.6665 | 0.976 | 1.195 
4| 0.756 | 1.126 | 1416 
5] 0.846 | 1.256 | 1.605 
6| 0.925 | 1.386 | 1.765 
7 
8 
9 
i) 


0.995 | 1.496 | 1.916 

1.066 | 1596 | 2.055 

1.126 | 1.696 | 2.186 

125 185 2.36 

20] 1.76 | 2565 | 336 

50[ 266 | 406 | 525 
100| 3.75 | 5.75 746 


But this approach does not take into account one thing: the 
curve narrows around the mean when N increases. There is 
another additive quantity: the variance. The standard 
deviation, square root of the variance, is proportional to YN 
and takes account of error compensations. 


We obtain a bell curve. A statistical theorem, called the 
limit central theorem, indicates that for N large the curve 
tends to a Gaussian. The range of a Gaussian is infinite for 
a finite standard deviation. 


We summarize the evolution of the uncertainty on the 
sum of N independent lengths measured with the same 
resolution 6 in a table. In italics, from N=10, these are 
numerical simulations carried out on a computer by 
generating random numbers, 
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‘The results of the measurements are often given with 
a confidence of 95%, which corresponds for a Gaussian to 
an uncertainty of about 20 . 


6. OTHER APPLICATIONS 


‘A runner wishes to measure his travel time. He has a 
watch with a digital display. ‘The watch shows that he starts 
at 10M52min and arrives at 11h 11min , The display is 
at the minute, so he starts between 10h52min00s and 
10h52min 59s . Hence the date of departure is within the 
interval t;=10h52min30s+30s | The resolution is one 
minute. The duration of the course is At=t,—t,. The 
results remain true for a difference. We have N=2 and 
At=19min-47s with 95% confidence. 


Same procedure if students measure an angular difference 
with a goniometer. Each measurement is within a minute 
of are so the uncertainty of the angular difference is of 47 
are seconds with a confidence of 95%. 


Seven persons are in an elevator. Its maximum load is 
S00kg. Their individual masses are measured with a 
resolution of one kilogram. The total mass is 499kg. What 
is the probability of being overloaded? 


For N=7 the uncertainty reaches one kilogram with a 


confidence of 80%. So there is a one out of ten chance for 
the elevator to be overloaded. 
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In the laboratory many measuring instruments have 
digital displays. The resolution is define by the last digit. 
But the overall uncertainty is much higher. It is necessary 
to consult the operating instructions of each device. 


CONCLUSION 


‘The general approach consists in combining laws of 
probabilities. The mathematical tool used is a change of 
variables, then one or more integrations, For the measure 
with a ruler described in this article, it was a sum of two 
independent random variables and we obtained a 
convolution. 

If one wants to do a faster calculation, an analysis of 
variance may be enough. We haye a random variable X 
that depends on N independent random variables X, : 
X=f(X,,X, Xv). We call 9; the 
standard deviation of X, and o that of X . For finite 
©, and small variations, we have the propagation of 
standard deviations formula iv}: 


(2L) 63 
a, 


‘And, independently of the probability distributions, this 
relation between the variances remains true. One can thus 
give its result with an uncertainty for 26. or 30. 

Is there a similar formula with the confidences? Yes, but it 
is approximate, it is the propagation of uncertainties 
formula: 
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x, 


with X= Xiny=AX,, f= finy +4 f and a constant 
confidence. This formula is very useful and allows a quick 
and reasonable calculation of the combined uncertainties. 
Moreover, it is exact if the form of the distributions is the 
same for X and the %,, For example, if the distributions 
X, are Gaussian, any linear combination is also Gaussian. 

We thus take account of compensations and avoid using the 


formula A f=))|0 f/Ox\Ax, which overestimates 


the uncertainties, sometimes even with such excess that one 
loses its physical sense. This last formula does not take into 
account any compensation, we have the worst situation, 
statistically unlikely. Here, for example, for N=100, one 
would have an uncertainty of 506 , instead of 5.75 in 
practice (95% confidence). 


In this article we focused on the resolution of an 
acquisition system that gives a discretization error. But one 
can also consider systematic errors and random errors. 
Here the ruler was supposed perfect, that means, accurate 
and precise. 
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VI. Mathematical Tools 


‘The mathematical tools used in chapter 1 are studied into 
the high school scientific section. The partial derivatives used in 
chapter 2 are taught during the first year of university but we 
quickly understand the link with the derivatives seen at the high 
school. It is at the end of the second chapter, with the use of 
matrices for generalized regression, that we immerse ourselves in 
university education, My goal is not to review or introduce all 
these concepts, only a small recap that can be useful to solve the 
exercises proposed. 


A - Derivatives 


flxte)-fix))| 


1- Definition: f ‘(x 


For example if f(x)=x 


(x+e) x +2xE+€ 


=2xe and f (x)=2x 


Ona graph the derivative corresponds to the slope of the curve at 
each point. 


2- Rules for basic and combined functions: 


function f [derivative f* | Af 
ax @ (constant) |Ax 
x* axe! A(x")=ox" A x 


cos( x) 


fcos( ax 
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cos( x) —sin(x) 

e e A(e*): 

In(x) A(In(x))=A x/|x} 

uty (wand v as functions of x) 
uy (product rule) 

7 (quotient rule) 

f(g(x)) )f'(g(x)) (chain rule) 


B - Partial derivatives 


A partial derivative of a function of several variables is its 
derivative with respect to one of those variables, with the others 
held constant. For example, consider the following function of 
three variables: 


F0sy.2) +4ay 


We can look at the variations of this function with respect to a 
variable while considering the other constants. We proceed then 
as for a derivative. So we have: 


The id "partial derivative of f with respect to 
x" treating y and ¢ like constants. d: curly d. 
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C - Taylor series 


With the notion of derivative we have studied the first-order 
behavior of a function around a point, here we refine to the 
higher orders, 

For every infinitely differentiable function and for «<1 we 


have the fling development in the neghbeshod of «pit 


Flxghe)=F lx) tef SL) HELA) SP) 


The more we take high order terms the better is the 
approximation. For example for f(x)=sin(x) and x,=0 we 


find: sin(e)~e— 


the same way cos e)=1—& 


3! 2 


Also: exp(c)=1+e — In(1+€ 


D- Integrals 


We show in maths that: 


For the standard normal distribution, we can verify that the mean 
is zero and that the standard deviation is equal to 1: 

and =f xp(x}dx=0 because the 
integral over a symmetric interval of an odd function is zero. 

J (-x)(-xe 7)dx then 

V2 02 


-f(-te 7 dx=0+ if dx 


so the standard deviation equals one, 
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We have used an integration by parts: 
A 


F 
Julx)v(x)de=[ulx)v(x)—f w'(x)v(x) dx 


then we did an integration by substitut 


We can go further by calculating the skewness 


w=] x pla)de. the kurtosis =f xp(x}dx and, in 


general, the moments of order n 41,= J x"-p(x) dx 


Alll these moments allow us to characterize a distribution. 


For a Gaussian ,.,=0 (symmetric). If this coefficient is 
negative the curve spreads to the left, if it is positive the curve 
spreads to the right. For a Gaussian B,=,/o"=3. If this 
coefficient is less than 3 the curve is more flat than a Gaussian, 
Fora binomial distribution : 


Integration by substitution: 


Let u=g'(x) be a new variable with g™' a continuous 


function strictly monotonic on [a,b] and g_ the inverse function, 


then: 
Jflxidx= J fg(u))g'(u)du 
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E-Series 
formula: (a+b)" > (rato 
ilk 


Binomi: 


Derivatives of geometric series: 


\ql<1_ then Xd 


1-q 


we take the derivative with respect to q: x kg 


a 7 
then Sik(k-a)qh?=—2, 
im (1-q) 
‘And finally we find the negative binomial formula: 
Sate 1).(k-ret)qh =H 


Ta 
ese 
(1-9) 2 7 


so 


A definition of the exponential function: e"=}° — 


- Gamma function rxj=f ele ‘dt 


This function is an extension of the factorial to real and complex 
number (except for 0, -1, -2...)"°. We will use it for half-integer 
numbers. We demonstrate with an integration by parts: 

T(x+1)=xP(x). P(1)=1 then for 1 integer 


T'(n+1)=n!. Moreover, P(1/2)=\7 allows to calculate the 


function for the half-integers. 
22For example m!=7.2 
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VII. Answers to Exercises 


Complete and High Quality Digital And Paper Book 
with all the exercises corrected on www.lulu.com 


‘To contact the author: ecrire@incertitudes.fr 


Chapter I : Random Variable 


EL: Ages Exercise on page 37 


15; mode=18; media 
range=4; standard deviatios 
‘mean deviation=0.844 


8; mean=18.333; geometric mean=18.303; 
113; root mean square deviation= 1.075; 


E2: Card Game  Etercise on page 37 


‘Number of possible draws: 32x31x30x29x28 divided by the different 
ways of arranging 5 cards 5x4x3x2x1=5! permutations then 201 376 
hands (32 choose 5). 


* Number of possible draws for a four aces hand: only one possiblity 
for the 4 aces, and 28 possibilities for the fifth card, so 28 possible 
hhands (one hand = 120 possible draws). 


Hence the probability p=28/20137 
of 7192=0,014%. 


39 chances on a million=1 out 


* For a flush: each color has 8 cards. Number of ways to choose 5 


alia) gs 
cards among 8 = 57g—sy = [8] = C8 = 56 


‘There are 4 colors : 4x56 =224. 


Hence the probability p = 224 / 201 376 = 1 out of $99 = 0,11 %. 
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Chapitre II : Correlation and Independence 


El: Correlations Exercise on page 86 


+0+0e0014141)) 
Ta ie XF9-H=VExIB so 
3/2~0.87 


i) 
ry ' 
1 a 
x 

¢) t=O. n=O. rel. Xyand X2 are not correlated. The same for Xs 


and Xs, Xoand Xs are dependents and totally correlated, 
Qa) K=O and X=0. O=1.22 and o=1.58 

€) ri:=0.904 . Quantities are positively correlated. 

Bea) XicOand x0, 1.73 and 0,153 

) ro=0 . Quantities completely uncorrelated, do not forget that the 
correlations sought here are linear. ‘There is a correlation in the form 
ofa. 


2-b) 3-c) 
3 3 
2 =f m2 Ll 
se = 
om ao = 
B2 git gy 2 3 
2 eg i 2} oo 
=m aaa 
“3 + 
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E2: Volumes Exercise on page 87 


100.1+100.0+99.9+100.0)/4 so Vi= 100.0 mL. 


OOO OAF HO 2 o.rmt then 0:~0.082mL 
a 5 


According to the central limit theorem and the distribution of the 
population Gaussian: A V=t.0/n=3.18x0.0816/2 

tages 9se=3.18 so AV, = 0.13 mL and AV; /V1~0.13/100 

‘The pipette, withe a confidence of 95%, has a uncertainty of 0.13 mL, 
so for 100 mL. a percent uncertainty of 0.13%. 


Vand 
Liviviiv-v. 


DIV\'V.'}=0.1x0+0%0.14{-0.1)x0+0%0.1=0 


é 


by definition 


so 1220, the quantities are totally uncorrelated and 
therefore independent. 


+0.1°+(=0.1)'+( 
ves 


O.1mL 


then Oy=0.115mL and AV~ 0.183 mL, AV WV = 0.09% 
4 
V(Vi, V2) then we have 


av 
av 


avie(S¥) avian AVZ=AV24AV2 


and AV=\3.AV,=0.18 Same result as in question 3- 
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Illustrations: 


3D representations in front of the table of contents and third thumbnail 
of the back cover. 


‘A random walker moves in a plane. He throws two coins and looks at 
the results: two tails, one tail and one head, one head and one tail or 
two heads. The first coin tells him if he has to make a first move 
towards the East or the West, the second if he has to make a second 
step towards the South or the North. ‘Thus at each time interval Ar he 
‘moves in the plane of two steps. During Ar the distance traveled by the 
walker is Ad = 2 p (p length of one step). 


At what distance from the starting point is the walker at the instant t? 
(after m time intervals: f= m A‘) 


. we draw the following tables: 


os[a[a las96| 2506 
os| 1 [a 
05 05 25% | 26% 


‘The center of the table is its starting point. On the abscissa x (direction 
East-West) the displacement is more or less one step (x=p 
and we have fixed Ad=/). Similarly on the y-axis: y=4p. In each 
square is indicated the number of possibilities to meet at this place. 
‘The second table indicates the probabilities (1/4, 4=2x2) 


In the four possibilities he is at a distance 12/2 from the point of origin. 
‘And, in terms of standard deviation, the characteristic distance is 
s=\(Oaly 816 


For n=2, we draw the following tables: 


aia 6% [1305] 636 
oj2 fale i [26 [ 23% 
2[2}t 6% [xs 66 
aon 
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For example to be at (x=0 ; y=1), there are two possible paths: 
(PE,PP) and (FP.PP), Standard deviation s,~1.033 
The walker has a one in four chance to be back to the starting point. 


For n=3, we draw the following tables: 


as{a[3]3[a 2% | 5% | 5% | 296 
os|3{9]9}3 5% [14% |14%6| 5% 
os|3[9[o[3 s,=1.234 
5% [14% |14%6| 596 
as{t[3]afi 
2% | 5% | 5% | 296 


2 [2] 4]e] 4] 2 | [osrla.coel2.a0.cn6lo.an| 
1 | 4 [26] 24/16 4 | | la-0e]9.an[o.26|2.654 
6 [24]26|24[ 6 
2.29.4] 14%4|9.4n6 2.39] 
a4 [16[24[a6[ 4 
2{a}alel 4] 1 | [sexes o.2rl6.20 6% 
24 12 10.4%6|1.636) 2.39% [2.696 0.404] 


lowing tables: 


25| 1] 5 |10/10| 6 0.90.50 2056.00.50. 


is [5 |25|50|50|25| 5 
05 |10| 50 [100 |100 | 50 | 10 
-05| 10 | 50 |100|100 | 50 | 10 
as| 5 | 25|50|50|25| 5 


0.52.40 4.90], 082.4040. 5] 


1.0640 9.209.064 996|1.0n] 


1.0649 9.2%|0. 0 4991.0] 


0.52.49 4.904, 082. 496|0.536] 


2s|1| 5 [10/10] 5 | 2 


0.190.504.0108 .59|0.196] 


25-15-05 05 15 25 


174 


For n=6, we draw the following tables: 


3 6 115 [20115] 6 | 2 | |ooyo.ssjo.4seo.sio.as6l0.1%6]0.0%) 
2 [6 [36 [90 [120] 90136] 6 | {os .o. aoe zoos 

1 [15 [20 90/15 
sept cela 57 a6 [220.4 

© [20 [120200 [e00 [200 [120] 20 
[ts }e0 Teas oolzos|o0T a5 | (0/2exfr.2.|7a53[osy] 
216 }36 [90 fzol oo [a6T « | [aat2-s.s3|7.a9 5.4/2.2 [0.4] 
-a [2 [oe [as [20 [as [6 [a | [oases ola ze[a. oa 
3-210 1 2 3 [ponfoswloaslasrlousefo.s)oos| 


Hence the evolution of the quadratic mean distance from the starting 
point as a function of time: 


tml olif2z]{[3[4[s5]e 
vt_[0.00 [1.00] 1.41 | 1.73 |2.00[ 2.24 [2.45 


Is (2p)] 0.000] 0.636] 2.033] 1.224] 2.417) 1.582) 1.722 


We plot the curves s as a function of t and s as a function of ¥¢ 


20 20 

15 ea va 15 

10 a 2 . 10 /- 

os os 

00 00 
o12a4567 00.05 1015 2025 30 


We find a much better correlation in Vb. Indeed we saw directly that the 
distance at the origin did not evolve proportionally to time, for n=2 we 
are about one unit of the origin, so we should be towards 3 for n=6. 
This variation in Vf is characteristic of diffusion phenomena and here 
finds its analogy with the compensation of errors in Vn. 
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IX. TABLES / Index 


A. Standard normal distribution 


Plz 


P(Z<~2)=: 


f flz)ae 


p(Z>z) 


(Z<z) 


Example 


P(Z>z) 


fle) 


-P(Z<z) 


P(Z-<1.67)~0.95254 
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B. Student's t-values 


otal Confidence (%) 
S0[_ 80] So] 95] 99] 99] ga5] sag] oa.) 
I 200|a08|_6aa| a2.7|_axa|_ea.7|_127| ane] _aa7| 
2[oa2| 189] 2.92| 420] 696992 14.1] 22.3] a1} 
3[076| 1642.35] a1a|_a5al_—saal_7as|_10.2] 125] 
‘a|ova15a]_2.1| 2.79] 2.75] .60]5.60| 7a] ai 
> [SLoral_aaa|_2.03| 257] a6] a0a|_a.77]_S.a0]_o.a7 
§ [ep oresaa] 194] 25} aaa] ara] a2 sail saad 
3 [“7[ovap tail 109] 239] 3.00] —a.50] a3] —a.79] sai] 
§ [Lov 1a0- 86 zai] 200226] anal 450] 5.09] 
& [9-070] 139 ras 220) za] a5] a9] _a30]_a78| 
5 [ao oof r37 rai 223] 276] 317] 3.50] 14] a9 
[aa 70] 1361802202 72| sir] a0] _a02|_aaal 
B [par aset tral aie saat sos] saa] aaa aaa 
2 [33010] 1a5[_a77| 216) 26s] aoa] _aa7] aas|_a22| 
@ [4a oso] z35| 1.76] 214] 2.62] 2.98] 2.39] 2.79] 4.14 
= [“a5[ oso] 134] 1.75] 2.23] 2.00] 2.95 3.20[ 37a] _ac07 
$ |[e(-oc9| 124s 76232] a50|_2.02| 226] 209] ai] 
E |_a7| 060] 123[_a7al_2ai] 257] 200] 222] a6s|_a.a7| 
 |te|_ 69] 123[_a7al 210] 256] _2.a0| 220] aca] aad] 
% [_19[ 069] 133[ 173| 209] 254 286] 317] 358] 3.89] 
2 [20] -o60[1.33|_1.72| 2.09] 2.59] 2.05| 2.15] ass] 2.09 
E [22] oso] 132172] 207] as] 202| a2] aso] 3.79) 
& [2a oar 33] ara] 2.09] 2.49]_2.00| 2.09] 2.47] 2.79 
= (25) ose tail L7a[ 20 24a] —27e| s07{ aaa] ar 
& [aol zai] a.70]_2.05| 2.47] 2.76] 2.05] .ai] 2.07 
3 [30[ oee[ rai] 170-209] 2a] 2.75] a3] 3.39] 305) 
= [aoe] 1.30] 2.60] 2.02] 2.49] 2.70] 2.07] ai] 2.59 
© | sol o.68] 1201.60] 2.01] 2.40] 2.60] 2.04] a2e] aso] 
$ [e068] 120.67] 2.00229] 2.60] 2.01] aza]_a.a0] 
& [70-8] 129] 1.97] 1 99] 2 29] 2.05] 2.90] 3.23] al 
8 [solos 129] 2.65] a.99] 2.37] 2.64] 2-69] 3.20) 3.02 
90[ 0.68] 1.29] 1.66[_1.99| 2.27] 2.63| ee] aaala.a0] 
To0| 068] 120] 1.66] 1.98] _2.20|_2.6a[ 27] ear] _a.a9| 
200|—o.ca] 1.29] 1.65| 1.97] 2.2] 2.60] 2.8q aa] a.aq 
‘300|_o.ca] 1.28] 1.65| 1.97] 22a] 2.50] 20a] 1a] 2.29 
S00|_o.67] 1.28] 1.65| 1.96] 2.29] 2.50] 2.09| aaa] a.ai 
F000] 0.67} 1.28] 1.65[_1.96| 2.23] _2s0| 2a] aio] a0] 
= [0.67] 2.20 1.6a[_2.96| 2.33] _2.50|_2.01] 3.09] 3.29] 
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C. Chi-square values 


x2 


Probability «to reject the hypothesis while itis true (%) 


%o] 90] 78] 80] 30] 20] a0] 8] 2] 3] 0 
[arses ees| pas aose azo [ae e| ars ner] ase] oa ae 
g [ep ser [eso] was] ate aaa] ame oro] oer] sez] ae 
§ [as[ aur von] so] cess afar] aoe] aaa] ose] avr] oe 
3B [aa —a06[ 779] t02| tea] se2| see] Baa] 2a7] 268] aaa] weT 
a Ear cai[ 10a[ 128] 16.34| 195] 21-6] 24.8] 27.6| 31.0] 334] 408 
9 [1s —7ar[_a08| a7] a7] 208] 228] 260| eas] al aaa] zea 
3 (eo, —~aze| aza[as5| asm | aea[ aso| aaa) aia| asa] ave[ asa 
2 rT wo0| 132| 163 20.34| 23.8| 262] 29.6) 327| 363] 3a5| 468 
3 [az asa] 140] a7.2| 2134] 249] 273] s08| a9] 37.7] aoa] an3 
2 [a 209 157] 190] 23.34| 27.1| 296| 33.2] 364] 40.3| 430/ 512 
Pes eas asa aaa eal aot s 
27] az ae |-aio] vee] sos seal ser] aoa] aaa] arf ose 
25| —aes| asa] aaa] soe] ses aes] soa] are] er] ase] oes 
30] aso] aos] aes] aose] ses] aes] 203] sae] eo] Sos] Bor 
32] —ies[ ara seal arse] ser[ asl wee] wea] soe] sas] ase 
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Bias rei bethertetheed feted 
Binomial distribution... 
Binomial formula. 
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Frequency. Seer 8 
Function of a continuous distribution... ALS 
Gamma function, 167 
Gaussian distribution.. 10,19 
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Geometric mean... 
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Inverse distribution... 
Inverse transform sampling. 
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Moment. e 
‘Negative binomial distribution... 
Nonlinear regression... 
Normal distribution, 
‘Numerical simulation, 
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Parabolic regression... hepsi 
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Polynomial regression, 7 
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Power of the test... 
Precision... 
Prediction.. 
Prediction Interval 
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Probability density functio 
Product of distributions... 
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‘Uncertainty calculations er) 
Uniform distribution. 107, 122, 153 
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This book is made for anyone interested in experimental 
sciences and mathematics. Statistics and surveys are very 
common in our society, and the application area is supposed 
to be as large as possible. We are willing to go beyond the 
theory so that the invistigator may find necessary tools to 
solve simple and rigourous uncertainties quantification. 
Indeed, science tries to link natural phenomena to a 
mathematical logic. Overall coherence and truth need to be 
practised by a critical mind that lays on measures 
accompanied with their uncertainties. 
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